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Preface 


For  the  twenty-ninth  year,  the  Research  and  Theory  Division  of  the  Association  for  Educational 
Communications  and  Technology  (AECT)  is  sponsoring  the  publication  of  these  Proceedings.  This  is 
Volume  #2  of  the  29th Annual  Proceedings  of  Selected  Papers  On  the  Practice  of  Educational 
Communications  and  Technology’.  This  volume  includes  papers  presented  at  the  national  convention  of  the 
Association  for  Educational  Communications  and  Technology  held  in  Dallas,  TX.  Copies  were  made 
available  to  Convention  attendees  and  AECT  members.  Volumes  1 and  2 are  available  through  the 
Educational  Resources  Clearinghouse  (ERIC)  system. 

This  volume  contains  papers  primarily  dealing  with  instruction  and  training  issues.  Papers  dealing  with 
research  and  development  are  contained  in  the  companion  volume  (29th  Annual,  Volume  #1). 

REFEREEING  PROCESS:  Papers  selected  for  presentation  at  the  AECT  Convention  and  included  in  these 
Proceedings  were  subjected  to  a reviewing  process.  All  references  to  authorship  were  removed  from 
proposals  before  they  were  submitted  to  referees  for  review.  Approximately  sixty  percent  of  the 
manuscripts  submitted  for  consideration  were  selected  for  presentation  at  the  convention  and  for 
publication  in  these  Proceedings.  The  papers  contained  in  this  document  represent  some  of  the  most  current 
thinking  in  educational  communications  and  technology. 
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Kiir§at  (J'agiltay 

Middle  East  Technical  University,  Turkey. 

Abstract 

Despite  more  than  30  years’  existence  of  computer  games  and  simulations  in  the  instructional  design 
movement,  available  comprehensive  design  paradigms  are  still  lacking  and  well-designed  research  studies  about 
the  question  of  “how  to”  incorporate  games  into  learning  environments  is  still  a question.  In  search  for  an 
answer,  this  study  explicates  the  third  generation  of  the  “FIDGE  model,”  which  is  an  instructional 
design/development  model  (IDDM)  that  has  come  to  life  as  a result  of  a formative  research  and  with  the 
inspiration  from  fuzzy  logic.  The  model  is  designed  for  creating  game-like  learning  environments,  educational 
or  commercial  games,  or  briefly,  video  games  in  general. 

Statement  of  the  Problem 

Traditional  instructional  design/development  models  (IDDMs)  have  been  criticized  on  the  grounds  that 
they  hardly  represent  a variety  of  structure,  although  they  are  abundant  in  number.  The  procedural  stratifications 
and  time-consuming  practices  have  constituted  the  main  rationale  of  these  criticisms.  On  the  other  hand, 
although  computer  games  and  simulations  have  a history  of  more  than  three  decades  in  the  instructional  design 
movement  and  computer  games  are  considered  as  powerful  tools  to  increase  learning  (Dempsey,  Lucassen,  et 
al.,  1996;  Dempsey,  Rasmussen,  et  al.,  1996);  there  are  two  major  problems  that  are  encountered.  One  is  that 
there  are  no  available  comprehensive  design  paradigms  and  the  other  is  the  lack  of  well-designed  research 
studies  (Gredler,  1996).  While  the  literature  is  growing  as  time  passes,  by  the  carbon  copy  researches  that  report 
perceived  student  reactions  preceded  by  vague  description  of  games  and  simulations  or  comparative  studies  of 
simulations  versus  regular  classroom  instruction  (Gredler,  1996),  the  question  of  how  to  incorporate  games  into 
learning  environments  rather  than,  simply,  to  master  the  material,  is  much  more  frequently  asked  to  the 
educational  researchers  (Dempsey,  Lucassen,  et  al.,  1996;  Dede,  1996).  In  conclusion,  there  is  the  apparent  and 
urgent  need  for  the  introduction  of  an  IDDM  that  will  help  and  guide  instructional  designers  and/or  game 
designers  for  the  efficient  use  of  games  and  simulations  in  educational  environments,  more  precisely  to  create 
game-like  learning  environments. 


Method  and  Data  Sources 

The  study  that  originated  the  model  was  designed  as  a special  methodology  that  is  similar  to  that  of  a 
case  study  method  of  qualitative  research  (Yin,  1996),  which  is  referred  as  “formative  research”  by  Reigeluth 
and  Frick  (1999,  p.  633).  Formative  research  methodology  fits  best,  while  the  researcher  is  interested  in  the 
design  and  development  process  rather  than  the  product  or  outcomes,  in  contextual  structure  rather  than  specific 
variables  and  in  discovery  of  the  underlying  elements  rather  than  conformation  (Merriam,  1998).  Moreover,  the 
reason  that  the  case,  which  was  selected  by  the  researcher,  was  not  especially  and  intentionally  designed 
according  to  a specific  model,  but  fulfills  the  same  goals  and  provides  the  same  context  as  researcher’s 
intention,  leads  the  design  of  the  study  towards  naturalistic  case  of  formative  research. 

Participants  of  the  study  consisted  of  18  out  of  56  senior  undergraduate  students  of  the  Computer 
Education  and  Instructional  Technologies  (CEIT)  Department  of  a university  in  a European  country,  with  the 
age  range  of  20  to  24.  Participants  enrolled  in  an  undergraduate  course  in  their  department,  were  invited  to 
participate  in  a study  that  was  designed  to  suggest  a new  instructional  design/development  model  that  might  be 
used  for  creation  of  game-like  learning  environments,  utilizing  a 3D  virtual  world  tool  on  the  Internet.  The 
researcher  collected  data  during  this  design/development  process,  duration  of  which  was  three  months. 

As  for  the  data  collection  and  analysis  techniques,  observations,  documents  (such  as  weekly  reports,  peer 
evaluations,  e-mail  logs  and  reflective  papers  about  the  specific  aspects  of  the  selected  case  written  by  the 
participants),  and  semi-structured  interviews  with  the  participants  (students),  instructors  (of  the  selected  case); 
and  content  analysis  were  used.  These  data  sources  were  the  ones  that  formative  research  implies. 

As  for  the  first  and  the  second  iteration  of  the  model,  the  same  undergraduate  course  was  selected,  in 
which  the  participants  used  the  first  and  the  second  generation  of  the  FIDGE  model,  respectively.  However, 
during  these  iterations,  the  selected  case  was  designed  to  fulfill  the  requirements  and  the  goals  of  the  FIDGE 
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model  and  to  provide  the  appropriate  context  as  researcher’s  intention,  leads  the  design  of  the  study  towards 
designed  case  of  fonnative  research.  During  the  first  iteration  the  researchers  interviewed  16  out  of  58  senior 
undergraduate  students  of  the  same  department,  with  the  age  range  of  20  to  23.  The  documents  that  are 
produced  by  the  students  were  also  collected  throughout  the  course  and  analyzed  afterwards.  As  for  the  second 
iteration,  the  researchers  prepared  different  questionnaires  for  each  phase  in  the  model  containing  open-ended 
items  and  collected  data  from  all  62  senior  students  enrolled  in  the  course.  In  the  light  of  the  analyses  of  the 
collected  data  the  model  revised  twice. 


Results  and  Conclusions 

Based  on  our  findings,  in  our  paper  we  put  forth  an  appropriate  and  complete  instructional 
design/development  model  for  the  creation  of  game-like  learning  environments.  In  accordance  with  many  of  the 
traditional  models,  our  model  also  consists  of  four  parts,  which  are  analysis,  design,  development  and 
evaluation.  However,  the  components  of  these  parts  and  the  way  they  are  structured  are  different  from  these 
traditional  models.  There  are  fuzzily  bounded  phases,  which  are  progressing  in  a dynamic  and  an  intertwined 
manner,  as  opposed  to  the  linear  structure  and  procedural  stratifications  that  follow  the  other.  For  these  reasons 
we  inspired  from  fuzzy  logic  (Zadeh,  1996),  which  is  an  enhanced  version  of  classical  logic  and  rests  on  the 
same  mathematical  foundations.  We  especially  interested  in  the  set-theoretic  facet  of  fuzzy  logic,  which  is 
concerned  with  fuzzy  sets,  whose  boundaries  are  not  sharply  defined.  Moreover,  by  ‘fuzzification’  which  is  the 
process  of  replacing  the  concept  of  a set  with  that  of  a fuzzy  set,  it  becomes  possible  to  provide  a way  of 
constructing  models  or  theories  that  are  more  general  and  more  reflective  of  the  imprecision  of  the  real  world 
than  the  models  or  theories  in  which  the  sets  are  assumed  to  be  sharply  bounded  and  definitely  limited.  This 
enlightened  exactly  what  we  want,  i.e.  the  proposal  of  an  alternatively  structured  IDDM  against  the  traditional 
IDDMs  that  have  been  criticized  for  their  linear  structures,  procedural  stratifications  and  time-consuming 
practices. 

During  the  first  iteration  the  emerging  issues  was  the  usability  problems  related  to  3D  environments 
and  the  collaboration-related  issues  that  was  articulated  by  the  participants  as  the  lacking  elements  in  the 
“design  and  development”  phase  of  the  model.  Considering  these  comments,  the  model  is  revised  and  evolved 
into  the  second  generation  of  the  FIDGE  model.  As  for  the  second  iteration,  the  emerging  theme  was  the 
emphasized  need  for  an  analysis  of  current  games,  by  the  participants,  to  be  able  to  find  out  the  game  utilities 
and  game  genres  appropriate  for  the  subject  matter  that  was  the  focus  of  the  intended  design  (Akilli  & Cagiltay, 
2006).  Throughout  the  course  of  the  study,  we  also  recognize  that  are  some  more  motivational  elements  that 
should  be  taken  into  consideration  while  designing  such  environments  other  than  the  already  present  ones  in  the 
model  which  were  a synthesis  of  various  researchers’  studies  such  as  Malone  (1980),  Malone  and  Lepper 
(1987),  Rieber  (1996),  Price  (1990),  Prensky  (2001),  Gredler  (1996),  Provenzo  (1992).  Thus,  via  the 
inspirational  study  that  Tuzun  (2004)  conducted,  we  also  add  some  more  elements  using  his  “Multiple 
Motivations  Framework.” 


The  FIDGE  Model 

General  Overview  of  the  FIDGE  Model 

In  the  current  study,  the  researchers  suggest  an  appropriate  and  complete  IDDM  for  the  creation  of 
game-like  learning  environments  based  on  the  findings.  In  accordance  with  many  of  the  traditional  models,  this 
model  also  consists  of  four  parts,  which  are  analysis,  design,  development  and  evaluation.  Additionally,  it 
possesses  an  additional  phase;  the  “pre-analysis”  phase.  However,  the  components  of  these  parts  and  the  way 
they  are  structured  are  different  from  the  traditional  models.  It  consists  of  dynamic  phases,  which  have  fuzzy 
boundaries  and  through  which  the  instructional  designers  progressed  in  a non-linear  manner.  Indeed,  the  reason 
of  these  characteristics  is  the  basis  of  the  model  that  is  cultivated  on  the  fuzzy  logic  concept,  which  also  leads  to 
a different  visualization  of  the  model  rather  than  the  traditional  “boxes-and-arrows”  approach  (see  Figure  1). 
Another  reason  is  that  the  model  is  proposed  depending  directly  on  actual  and  concrete  data  collected  from  real- 
life  practices.  What  is  more,  the  researchers  also  coined  the  below  presented  model’s  name  regarding  these 
attributes,  especially  the  non-linearity  and  fuzziness  emerged  from  the  findings.  It  is  called  as  “FIDGE”  model, 
which  is  the  acronym  that  stands  for  “Fuzzified  Instructional  Design  Development  of  Game-like  Environments” 
for  learning  (Akilli,  2004).  According  to  the  Oxford  English  Dictionary  (Oxford  English  Dictionary,  1989), 
“fidge”  as  a verb  means  “to  be  eager  and  restless;  to  express  pleasurable  eagerness  by  restless  movements,” 
which  is  also  consistent  with  the  impatience  that  anybody  shows  when  playing  a game  is  of  concern. 
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Figure  1.  General  structure  of  the  FIDGE  model. 


Before  scrutinizing  the  details  of  the  proposed  model,  it  would  be  better  to  emphasize  the  two  general 
patterns  that  were  dominant  within  the  findings.  First  one  is  the  contexts,  in  which  IDD  takes  place,  and,  in 
which  the  product  attained  at  the  end  of  IDD  process  will  be  used.  The  second  one  is  the  attributes  of  this  IDD 
process. 

The  first  pattern  is  both  a contributor  and  the  by-product  of  the  IDD  process.  It  seriously  affects  the 
quality  of  the  product  and  IDD  process  itself,  and  is  in  turn  affected  by  the  socio-organizational  needs  and 
cultural  issues,  which  appeared  during  the  IDD  process,  such  as  the  need  for  a leader  who  will  lead  the  rest  of 
the  team  throughout  the  process,  and  the  necessity  to  avoid  acting  with  their  emotions  and  feelings.  As  for  the 
context,  in  which  the  product  attained  at  the  end  of  IDD  process  will  be  used,  the  findings  revealed  the 
importance  of  the  appropriateness  of  the  product  regarding  the  socio-economic  status  and  the  abilities  of  the 
learner  with  consideration  of  the  cultural  issues. 

For  the  second  general  pattern,  the  “must-be”  non-linearity  and  dynamism  throughout  the  IDD  process; 
the  fuzziness  among  each  step  of  IDD  process;  and  lastly,  features  originated  and  inherited  from  games  and 
simulations  were  asserted  by  the  participants.  Throughout  the  entire  process,  all  the  participants  had  to  make 
modifications  and  revisions  in  their  plans  and  actions  that  they  took  to  overcome  the  problems  and  obstacles,  by 
means  of  continuous  evaluation.  Findings  have  indicated  that  it  is  impossible  to  omit  or  ignore  any  of  the 
analysis,  design,  and  development  phases  and  instead  of  isolating  these  phases  from  each  other  strictly  and 
conducting  them  in  a linear  sequence,  it  would  be  better  to  conduct  them  parallel  to  each  other  and  in  an 
intertwined  manner. 

Lastly,  findings  of  study  revealed  that  all  of  the  participants  used  some  features  peculiar  to  games  and 
simulations.  For  instance,  findings  of  the  study  provided  traces  of  unique  features  peculiar  to  simulations,  such 
as  non-linear  event  sequence,  intertwined  consequences  of  action-reaction  chains,  and  dynamic  set(s)  of 
relationships  changing  with  respect  to  the  actions  that  the  user  took. 

As  for  the  games,  findings  of  the  study  indicated  the  use  of  game  characteristics,  especially  in  the 
design  phase  of  IDD  process,  such  as  challenge,  fantasy,  curiosity  and  control  given  to  the  learners  that 
contribute  specifically  to  motivation  and  thus  eager  learning.  Moreover  the  findings  also  pointed  out  other 
features  of  games  and  simulations,  such  as  engagement,  interactivity  and  active  participation.  The  use  of  popular 
culture  elements  with  the  above  mentioned  elements  was  another  issue  revealed  by  the  findings  of  the  study. 
There  are  two  sets  of  principles  that  underlie  the  model,  which  are  related  to  some  socio-organizational  issues 
for  the  design  team  and  to  the  instructional  design/development  process  itself.  The  first  set  of  principles,  which 
is  related  to  peopleware  (soft)  and  technical  (hard)  issues,  is  as  follows: 

1 . Form  a multidisciplinary  and  multi-skilled  team  including  an  experienced  game -player 

2.  Provide  common  standards  about  the  work  done 

3.  Identify  and  develop  awareness  and  need  for  an  instructional  system,  and  create  mechanisms  for 

motivation. 

4.  Meet  the  need  for  a leader  and  a guide. 

5.  Establish  good  communication  strategies  and  create  active  involvement. 

6.  Manage,  plan  and  schedule  time. 


3 


The  second  set  of  principles,  which  is  related  to  the  whole  instructional  design/development  process 
itself,  is  as  follows: 

1 . Dynamic,  non-linear  and  fuzzy  phases. 

2.  Early  decisions  about  the  utilities  and  restraints  of  the  technology,  which  will  be  used  throughout  the 
project. 

3 . Analogous,  participatory  design  and  prototypes. 

4.  Support  from  the  literature. 

5.  Continuous  and  iterative  evaluation  and  synthesis. 

6.  Focus  on  the  modularity  and  flexibility  of  the  product. 

7.  Creativity. 

Lastly,  before  moving  on  the  details  of  the  model,  it  would  be  better  to  emphasize  that  the  components 
of  the  model  are  intertwined  through  each  other  and  sometimes  conducted  in  parallel  among  the  phases,  despite 
their  exhibition  as  a listing. 

Pre-analysis  Phase  of  the  FIDGE  Model 

The  reason  for  the  existence  of  this  phase  is  to  provide  a starting  point  for  the  instructional  designers  (see  Figure 
2).  Flowever,  if  there  is  no  need  for  such  a warm-up  period,  this  phase  could  be  skipped. 
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Figure  2.  The  visualization  of  pre-analysis  phase  of  the  FIDGE  model. 


All  of  the  following  issues  are  tentative  and  could  be  easily  changed  when  the  instructional  designers 
begin  to  conduct  analysis  phase: 

1 . Determine  and  specify  a tentative  target  group. 

2.  Select  a tentative  subject  regarding  your  target  group  and  depending  on  your  previous  experiences. 

3.  Conduct  a small  literature  review  (LR)  to  find  evidence,  whether  your  tentative  subject  fits  or  is  likely 
to  be  fit  for  creation  of  game-like  learning  environment,  or  not. 

4.  Specify  the  tentative  goals  of  your  design  according  to  the  selected  subject  and  regarding  your  target 
group. 

5.  Take  the  opinions  and  recommendations  of  the  subject-matter  experts,  and  a representative  group  of  the 
tentative  target  group  via  interviews. 

6.  Begin  to  explore  and  analyze  the  development  tool  /software. 

7.  Begin  to  analyze  different  games  to: 

a.  Differentiate  different  game  genres  such  as  strategy,  adventure,  sports,  etc. 

b.  Find  out  game  utilities  such  as  multiplayer,  collaboration,  communities  etc. 

c.  Find  out  which  game  genre  is  appropriate  for  which  subject  matter  e.g.  strategy  games  are 
appropriate  for  social  sciences. 

d.  Find  out  game  elements  such  as  the  use  of  pirates,  magic,  history,  etc. 

e.  Find  out  the  appropriateness  of  the  instructional  approach  in  relation  to  game  genre,  game  utilities, 
and  game  elements. 
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Analysis  Phase  of  the  FIDGE  Model 

In  this  phase,  needs  analysis,  learner  analysis,  context  analysis,  content  (or  task)  analysis,  cost  analysis  (if 
needed),  risk  analysis,  an  analysis  to  adjust  the  duration  and  the  frequency  of  the  system  for  effective  use,  and  a 
self-analysis  should  be  conducted,  while  the  tool  and  game  analyses  which  began  in  the  previous  phase  will 
continue  (see  Figure  3).  Moreover,  instructional  approach  and  its  implications  should  be  specified  and  a time 
planning  activity  should  be  done. 
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Figure  3.  The  visualization  of  analysis  phase  of  the  FIDGE  model. 


The  components  are  as  follows: 

1.  Needs  analysis: 

a.  potential  stakeholders’  (e.g.  teachers’,  students’,  parents’,  etc.)  attitudes  toward  computers 

b.  potential  stakeholders’  opinions  about  computer  use 

c.  potential  stakeholders’  expectations  from  simulations  and  games 

d.  why  the  target  group  should  use  simulations  and  games 

e.  the  insufficiencies  and  gaps  in  the  topic/  subject  stated  by  the  target  group  (continued  in  content 
analysis) 

f.  support  from  literature 

g.  obtain  or  create  sketches  illustrating  a completed  product  should  to  be  shown  to  the  target  group  or 
experts  as  an  example,  in  order  to  be  able  to  transfonn  an  abstract  concept  into  a tangible  one  for 
them. 

2.  According  to  the  conducted  needs  analysis,  the  general  goals  of  the  project  will  be  constituted;  more 
precisely,  the  needs  will  be  transformed  into  the  general  goals  of  the  project. 

3.  Learner  analysis  (conducted  parallel  to  needs  analysis  and  includes  real  observations,  surveys, 
structured  or  semi-structured  interviews  with  the  actual  target  group  and  time  schedule  for  all  these): 

a.  Actual  target  group’s  background  information,  i.e.  characteristics,  attributes,  skills,  prior 
knowledge,  and  specific  entry  competencies 

b.  Support  and  help  from  the  literature 

4.  Context  analysis  (conducted  parallel  to  needs  and  learner  analyses): 

a.  Actual  learners’  perspectives  about  the  attributes  of  a game-like  learning  environment,  in  which 
they  would  learn  the  designated  content. 

b.  The  role  of  the  teacher  or  instructor. 

c.  The  amount  of  the  learner  control. 
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d.  Examination  of  the  computer  infrastructure  (fulfillment  of  the  necessary  and  sufficient  conditions, 
specification  of  minimum  system  requirements  to  work  out  the  prepared  program,  identification  of 
the  hardware -related  issues). 

e.  Specification  of  the  socio-economic  status  of  the  learners  (in  relation  with  their  computer  literacy, 
in  order  to  determine  at  which  grade  the  program  will  be  used). 

f.  Begin  to  lay  design  foundations. 

5.  Content  (task)  analysis  (conducted  parallel  to  needs,  learner  and  context  analyses;  and  affected  by  tool 
analysis): 

a.  Efficiency  assessment  that  stood  for  the  optimum  amount  of  content  in  a limited  amount  of  time. 

b.  Checking  the  accuracy  of  the  content  (fonnative  evaluation  with  experts  and  learners). 

c.  Verifying  the  topics  included  in  the  content  through  various  resources  (literature  review). 

d.  Taking  students’,  subject  matter  experts’  and  experienced  instructional  designers’  opinions 
(formative  evaluation). 

e.  Carrying  out  step-by-step  reduction  (regarding  the  limitations  and  boundaries  of  the  tool  and  the 
structure  (or  nature)  of  the  tool)  (iterative  cycles  of  formative  evaluation). 

f.  Setting  the  structure  of  the  content  (regarding  the  limitations  and  boundaries  of  the  tool  and  the 
structure/nature  of  the  tool). 

g.  Creating  the  main  elements  of  the  scenario. 

h.  Synthesis  of  the  collected  opinions  of  the  students  and  the  experts;  the  elements  included  in  the 
content;  and  the  instructional  designers’  own  opinions  (to  provide  intact  objectivity). 

6.  Tool  analysis  (began  in  the  pre-analysis  and  will  be  continued  in  analysis). 

a.  The  tool’s  structure/nature: 

a.  What  its  uses  are. 

b.  How  it  is  used. 

c.  What  its  limitations  and  utilities  are. 

d.  Students’  or  learners’  viewpoints  and  reactions  to  the  tool. 

e.  Investigation  for  alternatives  to  the  selected  tool/  technology. 

b.  Thinking  of  suggestions  about  updating  and  maintenance  of  the  system  regarding  tool  analysis. 

c.  Thinking  of  the  issues  concerning  the  guidance  for  and  support  to  the  user,  such  as  ‘help,’  or 
‘technical  support.’ 

7.  Game  analysis  (began  in  the  pre -analysis  and  will  be  continued  in  analysis,  it  is  conducted  parallel  to 
tool  analysis  and  in  relation  to  learner,  content,  context  analyses  and  instructional  approach): 

a.  Think  of  game  utilities  such  as  multiplayer  options,  collaboration,  online  virtual  communities. 

b.  Think  of  which  game  genre  is  appropriate  for  your  instructional  design/development. 

c.  Think  of  the  game  elements  for  your  instructional  design/development. 

d.  Specify  the  appropriateness  of  the  instructional  approach  in  relation  to  game  genre,  game  utilities, 
and  game  elements  of  your  instructional  design/development. 

8.  Instructional  approach  (selected  regarding  the  structure  of  the  content,  tool’s  nature/structure): 

a.  Selection  of  instructional  approach  (e.g.  discovery  learning,  scenario-based  learning,  problem- 
based  learning  or  a hybrid  approaches,  which  are  offspring  of  two  or  more  different  approaches). 

b.  The  implications  of  the  selected  instructional  approach  to  the  design. 

c.  Assurance  of  the  selected  instructional  approach’s  capacity  and  aptitude  for  the  application  of 
game-elements. 

d.  Adjustment  of  the  instruction’s  duration  and  frequency  for  effective  use. 

e.  Fonnative  evaluation  to  take  the  opinions  of  the  learners  and  the  experts. 

9.  Self-analysis  for  each  instructional  designer  in  the  design  team: 

a.  Find  out  the  needs,  characteristics  and  skills  that  are  lacking,  but  should  be  possessed  by  the 
members  in  the  design  team. 

b.  Specify  strategies  to  gain  them. 

10.  Risk  analysis: 

a.  Envision  of  potential  risks. 

b.  Outline  of  a “panic  room”  plan  against  these  foreseen  risks. 

c.  Cautions  both  to  avoid  and  to  solve  possible  problems. 

11.  Time  schedule: 

a.  Time  arrangement  for  group  meetings. 

b.  Time  arrangement  for  meetings  with  the  designated  experts. 

c.  Time  arrangement  for  meetings  with  learners  from  the  target  group. 

12.  Cost  analysis  (if  needed): 

a.  Preparation  of  an  appropriate  budgetary  plan  regarding  the  variables  emerged  throughout  the 
above  analyses. 
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13.  Preparation  for  design: 

a.  Specifying  design  foundations  of  game-like  learning  environment 

b.  Writing  down  the  tentative  design  decisions  as  the  design  team  envisions  them  at  the  moment,  i.e. 
a very  general  overview  of  design. 

c.  Writing  down  the  ‘scenario  bits,’  such  as  the  main  theme  of  the  scenario,  the  characters,  etc. 

d.  Drawing  or  providing  tentative  sketches  illustrating  an  example  product  (The  design  team  could 
find  an  example  or  create  one  to  be  used  in  formative  evaluations  mainly  to  give  an  idea  to  the 
interviewees  about  how  the  completed  product  would  look  like). 


Design-Development  Phase  of  the  FIDGE  Model 

In  this  phase,  scenario  preparation;  content  clear-cuts;  specification  of  motivation,  attention,  feedback,  and 
learning  assessment  elements;  preparation  of  user-help;  creation  of  prototypes;  preparation  of  rating  scales, 
checklists  and  interview  guides  for  formative  evaluations;  design  of  orientations;  and  insurance  of  usability 
issues,  product’s  modularity  and  flexibility  will  be  conducted  (see  Figure  4). 
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Figure  4.  The  visualization  of  design-development  phase  of  the  FIDGE  model. 


It  would  be  better  to  emphasize  that  the  implementation  phase  of  the  traditional  models  is  contained  in 
the  intertwined  design  and  development  phase  of  the  FIDGE  model.  This  phase  encompasses: 

1.  Preparation  of  more  than  one  scenario,  namely  alternative  scenarios  for  the  game-like  learning 
environment  and  selection  of  the  most  appropriate  one: 

a.  Regarding  the  content  analysis. 

b.  Regarding  the  selected  instructional  approach. 

c.  Regarding  the  boundaries  of  the  tool. 

d.  Utilizing  the  team  member’s  wide  experiences  as  a game  player. 

This  gives  an  opportunity  to  switch  to  another  scenario,  in  case  the  scenario  prepared  at  the  beginning 
failed  to  be  implemented. 

2.  Preparation  of  the  scenario-related  components: 

a.  Setting  of  the  scenario  (prototypes  should  be  prepared,  to  be  used  to  take  feedback  from  the 
learners  and  ID  experts  continuously  (formative  evaluation)). 

b.  Plot  structure  of  the  scenario  (a  typical  use  case  should  be  written,  which  also  provides  guidance 
for  the  usage  of  the  prepared  program). 
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c.  A flowchart  regarding  the  scenario  (the  flowchart  should  inherit  the  content’s  structure  and  should 
be  framed  by  the  tool’s  limitations). 

3.  Content  clear-cuts  (as  an  extension  of  the  content  analysis  in  the  analysis  phase): 

a.  Clarify  the  content  in  its  brief,  intertwined  and  clear-cut  form  using  the  continuing  step-by-step 
reductions  and  modifications  via  iterations  and  feedbacks  from  the  team  members;  subject  matter 
experts,  ID  experts;  and  learners  (formative  evaluation). 

b.  Make  necessary  changes  in  the  goals,  content  analysis  and  the  flowchart  according  to  these 
modifications  and  reductions. 

4.  Specification  of  motivation,  attention,  feedback,  learning  assessment,  interaction  and  engagement 
elements  for  creation  of  game-like  learning  environments: 

a.  Utilize  the  essential  elements  of  many  commercial  games  possess,  such  as  curiosity,  challenge, 
fantasy  and  control  given  to  the  learner  to  give  his/her  own  decisions. 

a.  Employ  multimedia  features,  such  as  animations  for  the  introduction  of  your  storyline  in  the 
scenario  or  throughout  the  course  of  your  game  (Tuzun,  2004). 

b.  Use  rewards,  such  as  awards,  points,  badges,  symbolic  cups,  etc.  (Tuzun,  2004). 

c.  Create  opportunities  for  the  user  to  be  able  to  select  or  create  his/her  own  fantastic/realistic 
identity,  avatar,  username,  etc.  (Tuzun,  2004). 

d.  Specify  the  elements  that  will  make  your  system  unique  in  terms  of  learning,  socialization, 
creativity,  etc.  (Tuzun,  2004). 

b.  Utilize  elements  from  popular  culture. 

c.  Pay  attention  to  the  relatedness  of  motivation  elements  with  the  feedback  and  attention 
components. 

d.  Feel  free  to  combine  feedback  and  the  learning  assessment  elements. 

e.  Pay  attention  to  interaction  and  engagement  elements,  which  are  also  peculiar  to  games,  related  to 
the  feedback,  motivation  and  content. 

f.  Include  activities  to  provide  learners’  active  engagement. 

g.  Enrich  the  social  aspect  of  the  interaction  provided  via  the  program  you  are  designing,  to  help  your 
learners  to  establish  a virtual  community,  or  to  give  the  feeling  of  togetherness  via  employing  chat, 
e-mail,  etc.  features. 

h.  Specify  collaboration  utility  and  climate  of  your  game-like  learning  environment  by  describing 
how  the  users  will  collaborate,  what  utilities  they  will  have  when  collaborating  each  other  (e.g. 
will  they  share  the  tasks,  complete  only  their  own  part  of  the  task  and  merge  each  completed  part 
at  the  end  or  each  of  them  will  complete  the  task  on  their  own  and  use  collaboration  utility  of  your 
program  to  get  help  from  each  other?). 

The  employment  of  formative  evaluations  to  specify  the  details  and  components  of  the  above  is 
also  important  and  helpful. 

5.  Preparation  of  user-help  which  should  be  conducted  parallel  to  tool  analysis  including: 

a.  The  issues  concerning  guidance  for  user  such  as  ‘help.’ 

b.  The  issues  concerning  support  for  user,  such  as  ‘technical  support.’ 

6.  Creation  of  prototypes  (in  relation  to  the  above  mentioned  analyses  and  scenario) 

a.  Paper-based  prototypes  preparation. 

b.  Computer-based  prototypes  preparation. 

These  prototypes  are  used  to  take  feedback  from  the  learners,  experts  and  team  members,  about  both 
the  user-interface  design  and  the  overall  design  itself.  They  are  also  likely  to  reveal  the  above-given  issues  about 
the  motivation,  attention,  feedback  and  the  learning  assessment  elements  of  the  design.  All  the  way  through 
these  feedbacks,  it  is  likely  that  the  details  of  the  ‘user-help,’  or  ‘technical  support  will  also  emerge. 

7.  Preparation  of  rating  scales,  checklists  and  interview  guides  for  formative  evaluations  by: 

a.  Including  items  about  the  arrangement,  presentation,  appropriateness,  consistency  of  the  content. 

b.  Including  items  about  the  general  appearance,  appropriateness  and  consistency  of  the  user- 
interface. 

c.  Including  items  about  the  extent  to  which  the  program  appeals  to  the  user. 

8.  Design  of  orientations,  such  as: 

a.  An  orientation  about  your  program  to  avoid  misunderstandings. 

b.  A more  general  orientation  to  acquire  your  target  audience  with  the  basic  computer  literacy  and 
game -related  skills  (such  as  adjustment  of  an  environment,  in  which  the  learners  could  play  a 
simple  game  to  acquire  the  game-logic  and  gain  basic  eye -hand  coordination). 

9.  Insurance  of  usability  issues  throughout  the  whole  phase  by: 

a.  Keeping  in  mind  that  everything  should  be  user-centered  and  check  your  program’s  status 
accordingly,  since  they  would  be  the  ones  who  would  use  the  product. 

b.  Being  aware  of  usability  issues  and  employing  them  in  the  first  place. 


10.  Assurance  of  product’s  modularity  and  flexibility  by  providing  as  much  flexibility  and  modularity  as 
possible  for  the  final  product,  so  that  the  need  of  a radical  change,  which  might  emerge  following  the 
formative  evaluations,  could  easily  be  conducted. 

Lastly,  each  of  the  above-mentioned  elements  should  be  supported  by  the  literature.  It  should  also  be 
emphasized  that  final  user  interface  could  not  even  bear  a resemblance  to  the  initial  one  envisioned  at  the 
beginning.  What  is  more,  it  is  a possibility  to  be  confronted  with  a very  different  version  of  the  program, 
compared  to  the  previously  visualized  design  at  the  beginning. 


Evaluation  Phase  of  the  FIDGE  Model 

Although  the  related  elements  of  the  evaluation  phase  were  presented  in  the  above  phases,  it  would  be  better  to 
give  the  general  structure  and  main  elements  of  the  evaluation  phase.  Evaluation  phase  has  three  main  elements, 
which  are  formative  evaluation;  summative  evaluation  and  the  synthesis  (see  Figure  5). 

Instructional  designers  should  never  forget  that  evaluations  and  feedback  taken  from  the  learners 
should  be  continuous  and  should  begin  as  soon  as  they  started  with  the  pre-analysis  (or  alternatively,  analysis) 
phase. 

Before  conducting  evaluations,  the  instructional  designers  should  clarify  the  issues,  such  as,  by  whom  the 
product  would  be  evaluated,  how  they  would  be  reached,  where  the  evaluations  would  be  conducted.  For  data 
collection,  instructional  designers  should  employ  rating  scales,  checklists  and  the  interview  guides  that  would  be 
already  prepared  in  the  design/development  phase. 

Instructional  designers  should  conduct  the  formative  evaluations  frequently  and  with  shorter  intervals 
throughout  the  instructional  design/development  process  and  should  employ  them  while  determining  the 
foundation  stones  of  the  instructional  design/development  process. 

Instructional  designers  should  conduct  formative  evaluations  with  the  team  members,  their  peers, 
learners  in  their  target  group  and  various  experts  of  various  professions;  however,  as  stated  in  the  previous  parts, 
the  learners  representing  the  variance  of  the  target  group  should  be  in  the  first  place.  This  also  puts  forth  the 
usability  test  that  should  be  conducted  within  the  evaluation  phase. 
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Figure  5.  The  visualization  of  evaluation  phase  of  the  FIDGE  model. 


In  the  synthesis  part,  as  its  name  implies,  instructional  designers  should  make  a synthesis  and 
interpretation  of  all  the  data  collected  from  the  evaluations,  related  literature  and  their  own  comments,  when 
making  the  final  decisions  about  the  project.  For  these  purposes,  after  each  evaluation,  collected  data  should  be 
analyzed;  common  points  should  be  noted  and  discussed  with  the  other  instructional  designers  in  the  team. 

As  for  the  summative  evaluation,  it  will  be  used  to  evaluate  the  instructional  system  as  a whole.  Flowever, 
summative  evaluation  is  not  as  critical  as  it  was  for  the  traditional  model,  since  there  will  not  be  much  left,  due 
to  the  continual  formative  evaluations  conducted  throughout  the  design-development  phase. 

Summary  of  the  FIDGE  Model 
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To  summarize,  the  FIDGE  model  is  a real-life  originated  model  that  has  a dynamic,  non-linear  and  fuzzy 
structure  and  is  enriched  by  unique  features  of  games  and  simulation,  which  combines  the  context  with 
peopleware  throughout  the  instructional  design-development  process. 

The  proposed  model  might  be  used  in  creation  of  educational  games  as  well  as  in  creation  of  game-like 
learning  environments.  The  researchers  think  that  the  proposed  model  might  be  appropriate  to  be  used  by  both 
novice  and  expert  users.  The  existence  of  the  “pre-analysis”  phase  of  the  model  is  the  most  apparent  evidence 
that  this  model  addresses  the  novice  instructional  designers’  needs.  Another  evidence  is  that  they  are  the 
affiliates  of  the  so  called  “game  generation,”  who  are  different  from  many  of  us  in  various  aspects  and  possess 
differentiating  characteristics  and  skills  resulting  from  different  experiences  and  the  “new  media  society” 
(Prensky,  2001,  p.65;  Calvert  & Jordan,  2001).  However,  they  also  lack  sufficient  instructional  design 
experience,  which  would  have  impact  on  their  use  of  this  model,  both  positively  and  negatively.  The  probable 
positive  effects  are  their  untouched  creativity  and  ingenious  design  habits.  The  probable  negative  effects  would 
be  the  difficulty  in  understanding  the  model  or  misinterpretations,  which  would  result  in  void  and  ineffective 
design  practices. 

As  for  the  expert  instructional  designers,  the  researchers  believe  that  this  model  might  widen  their 
visions  and  help  them  catch  up  with  the  current  trends  and  changes  of  the  coming  generation. 

Lastly,  among  the  limitations  of  the  model,  the  probable  inheritance  from  the  selected  case  and  the 
complex  and  complicated  nature  of  the  model  could  be  asserted.  The  elements  that  constituted  the  model 
seemed  to  be  affected  by  the  IDD  model  used  in  the  case.  For  instance,  the  use  of  prototypes  was  inherited  from 
the  “rapid  prototyping”  model.  However,  it  would  be  meaningless  to  strive  naming  this  concept  in  another  way 
or  to  eliminate  it,  since  it  was  found  to  be  very  useful.  As  for  the  complex  and  complicated  nature  of  the  model, 
it  could  be  said  that  this  is  the  first  impression  and  it  would  be  much  easier  to  use  this  model  due  to  its  more 
flexible  structure  and  fitting  nature  to  human  reasoning  than  other  traditional  models. 

In  conclusion,  it  should  be  kept  in  mind  that  all  these  issues  are  personal  assumptions  and  visions  of  the 
researchers  and  should  be  verified  by  the  follow-up  studies.  It  would  be  hardly  possible  to  clarify  the  uses,  users 
and  the  limitations  of  the  model,  earlier  than  the  conduction  of  such  follow-up  studies. 


Table  1 

Summary  of  the  FIDGE  Model 
Issue  Its  Property 

Participants  All  of  actively  participating  learners  and  experts 
Team  Multidisciplinary,  multi-skilled,  game -player  experience 

Environment  Socio-organizational,  cultural 

Process  Dynamic,  non-linear,  fuzzy,  creative,  enriched  by  games’  and  simulations’ 

elements  (fantasy,  challenge,  etc.) 

Change  Continuous,  evaluation-based 

Evaluation  Continuous,  iterative,  formative  and  summative,  fused  into  each  phase 

Management  Need  for  a leader  and  a well-planned  and  scheduled  time  management 
Technology  Suitable,  compatible 

Use  By  (novice  /expert)  instructional  designers  and  educational  game  designers  for 

game-like  learning  environments  and  educational  games 
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Web-based  tests  in  Second/Foreign  Language  Self-assessment 
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Abstract 

This  article  focuses  on  the  use  of  computers  in  Language  Testing.  The  first  part  of  the  article  gives  a 
general  introduction  to  three  different  kinds  of  tests  administered  with  a computer:  (1)  computer-based  tests  (CBTs), 
(2)  computer-adaptive  tests  (CATs),  and  finally  (3)  web-based  tests  (WBTs).  On  the  other  hand,  the  second  part  of 
the  article  focuses  on  self-assessment  and  web-based  tests,  presenting  the  DIALANG  project  (a  new  European  work 
which  has  developed  diagnostic  language  assessment  tools  in  fourteen  European  languages,  delivered  over  the 
Internet). 


Introduction 

Technology  is  increasingly  being  an  important  factor  of  change  in  education.  As  a matter  of  fact, 
instructors,  administrators,  students,  parents  and  book  editors  and  writers  have  experienced  during  the  last  two 
decades  the  greatest  revolution  that  ever  happened  in  the  classroom.  This  advancement  has  been  influenced  basically 
by  the  introduction  and  use  of  the  computer  in  the  process  of  teaching  and  learning.  Thus,  learning  became 
significantly  richer  as  students  started  having  access  to  new  and  different  types  of  information,  as  they  manipulated 
it  on  the  computer  through  graphic  displays  or  controlled  experiments  in  ways  never  before  possible,  and  as  they 
could  be  able  to  communicate  their  results  and  conclusions  in  a variety  of  media  to  their  teacher,  students  in  the  next 
classroom,  or  students  around  the  world. 

The  rationale  behind  Foreign  and  Second  Language  instruction  is  based  on  the  notion  that  maximum 
contact  with  the  target  language  yields  maximum  learning,  which  is  why  this  field  has  found  a great  advantage  in  the 
introduction  of  computers  in  the  classroom  (Council  of  Europe  1997).  The  new  approach  allowed  addressing  all  four 
critical  skills  in  language  learning  (speaking,  listening,  writing,  and  reading)  by  engaging  the  students’  interest  and 
providing  a culturally  and  linguistically  realistic  context,  at  the  same  time  that  more  materials  were  accessible. 

Computer-based  tests  (CBTs) 

Language  Testers  also  found  a new  perspective  for  the  assessment  and  evaluation  of  language  acquisition. 
In  fact,  CBTs  started  being  developed  and  used  since  the  early  80s.  These  tests  represented  a new  way  of 
measurement  for  language  competency,  and  were  delivered  on  an  individual  computer  or  a closed  network.  CBTs 
can  be  offered  at  any  time  unlike  mass  paper-and-pencil  administrations  which  are  constrained  by  logistical 
considerations.  In  addition,  CBTs  consisting  of  dichotomously-scored  items  can  provide  feedback  on  the  test  results 
immediately  upon  completion  of  the  test.  They  can  also  provide  immediate  feedback  on  each  test  taker's  responses,  a 
characteristic  that  is  very  useful  for  pedagogical  purposes  (Bennett  1999).  Flere  we  highlight  some  of  the  main 
advantages  found  in  CBTs: 

• Efficient  Administration:  CBTs  can  be  administered  to  individuals  or  small  groups  of  students  in 
classrooms  or  computer  labs,  eliminating  timing  issues  caused  by  the  need  to  administer  paper/pencil  tests 
in  large  groups  in  single  sittings.  Different  students  can  take  different  tests  simultaneously  in  the  same 
room. 

• Immediate  Results:  One  of  the  major  drawbacks  of  testing  on  paper  has  been  the  long  wait  for  results 
because  of  the  need  to  distribute,  collect,  and  then  scan  test  booklets/answer  forms  and  hand  score  open- 
response  items  and  essays.  On  the  other  hand,  the  results  of  computer-based  tests  can  be  available 
immediately,  providing  schools  with  diagnostic  tools  to  use  for  improved  instruction,  and  states  with 
information  to  guide  policy.  Even  open-ended  items  can  be  scored  automatically,  greatly  reducing  cost  and 
scoring  time  (Thompson  1999). 

• Efficient  Item  Development:  As  computer-based  testing  becomes  more  developed,  item  development  will 
be  more  efficient,  higher  quality,  and  less  expensive  (National  Governors  Association  2002).  Bennett 
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(1998)  believes  that  at  some  point  items  might  be  generated  electronically,  with  items  matched  to  particular 
specifications  at  the  moment  of  administration. 

• Increased  Authenticity:  Computers  allow  for  increased  use  of  “authentic  assessments” — responses  can  be 
open-ended  rather  than  just  relying  on  multiple  choice. 

Computer-adaptive  tests  (CATs) 

Adaptive  testing  is  not  new.  In  fact  it  has  probably  been  around  for  centuries  as  a better  way  to  test.  And 
even  since  the  beginning  of  the  20th  century,  when  large-scale  testing  began,  adaptive  tests  were  some  of  the  first 
tests  ever  constructed.  Computers  as  a testing  medium  rapidly  attracted  the  attention  of  psychometricians  because 
they  allow  the  application  of  Item  Response  Theory  (IRT)  for  delivering  CATs,  which  can  often  pinpoint  a test 
taker's  ability  level  faster  and  with  greater  precision  than  paper-and-pencil  tests.  The  computer  can  make  the 
necessary  calculations  needed  to  estimate  a person’s  proficiency  and  to  choose  the  questions  to  present.  Based  on 
the  test  taker's  responses,  the  computer  selects  items  of  appropriate  difficulty  thereby  avoiding  delivering  items  that 
are  too  difficult  or  too  easy  for  a test  taker,  but  instead  selects  more  items  at  the  test  taker's  level  of  ability  than  a 
non-adaptive  test  could  include  (Alderson,  Clapham  & Wall  1995;  McNamara  1996;  Fernandez  Alvarez  and  Sanz 
Sainz  2004).  CATs  “offers  a potentially  more  efficient  way  of  collecting  information  on  people’s  ability”  (Hughes 
2003:  23).  Figure  1 shows  in  a clear  and  detailed  manner  the  way  CATs  work: 


ITEM  BANK,  ORDERED  BY  DIFFICULTY 

C$  C$  C3  P 

EASIER  MORE  DIFFICULT 


Figure  1.  Selection  of  items  in  a CAT 

According  to  Davies  et  al.  (1999:  29),  CATs  “are  claimed  to  facilitate  greater  accuracy  of  measurement  as 
a result  of  presenting  candidates  with  items  which  are  at  the  maximum  discretion  level,  i.e.,  are  more  or  less  at  the 
candidate’  level  of  ability,  items  of  this  type  providing  more  information  about  the  candidate  than  items  which  are 
too  easy  or  too  hard.” 

CATs,  like  CBTs,  show  some  advantages.  In  fact,  CATs  were  developed  to  eliminate  the  time-consuming 
and  traditional  test  that  presents  easy  questions  to  high-ability  persons  and  excessively  difficult  questions  to  low- 
ability  testees.  Dunkel  (1999)  identifies  some  other  advantages  in  their  use: 

• Self-Pacing:  CATs  allow  test  takers  to  work  at  their  own  pace.  The  speed  of  examinee  responses  could  be 
used  as  additional  information  in  assessing  proficiency,  if  desired  and  warranted. 

• Challenge:  Test  takers  are  challenged  by  test  items  at  an  appropriate  level;  they  are  not  discouraged  or 
annoyed  by  items  that  are  far  above  or  below  their  ability  level. 

• Immediate  Feedback:  The  test  can  be  scored  immediately,  providing  instantaneous  feedback  for  the 
examinees. 
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• Improved  Test  Security:  The  computer  contains  the  entire  item  pool,  rather  than  merely  those  specific  items 
that  will  make  up  the  examinee's  test.  As  a result,  it  is  more  difficult  to  artificially  boost  one's  scores  by 
merely  learning  a few  items  or  even  types  of  items  (Wainer  1990).  However,  in  order  to  achieve  improved 
security,  the  item  pool  must  be  sufficiently  large  to  ensure  that  test  items  do  not  reappear  with  a frequency 
sufficient  to  allow  examinees  to  memorize  them. 

• Multimedia  Presentation:  Tests  can  include  text,  graphics,  photographs,  and  even  full-motion  video  clips, 
although  multimedia  CAT  development  is  still  in  its  infancy. 

Web-based  tests  (WBTs) 

The  last  innovative  testing  technique  is  the  so-called  WBTs,  which  is  a CBT  delivered  via  the  internet, 
written  in  the  "language"  of  the  internet  (HTML),  and  possibly  enhanced  by  scripts.  The  test  is  located  as  a website 
on  the  tester's  server  where  it  can  be  accessed  by  the  test  taker's  computer,  the  client.  The  client's  browser  software 
(Netscape  Navigator,  MS  Internet  Explorer)  displays  the  test,  the  test  taker  completes  it  and  -if  so  desired-  sends 
his/her  answers  back  to  the  server,  from  which  the  tester  can  download  and  score  them.  If  the  test  consists  of 
dichotomous  items  (true/false  or  multiple  choice)  it  can  be  made  self-scoring  by  using  scripts. 

Even  though  we  can  still  find  several  drawbacks  in  the  use  of  WBTs  (like  security  or  technical  problems 
such  as  server  failure  and  browser  incompatibilities),  it  is  obvious  that  there  are  several  advantages  linked  to  them. 
Roever  (2001)  distinguishes  three  main  advantages: 

• Flexibility  in  time  and  space:  This  is  probably  the  biggest  advantage  of  a WBT.  All  that  is  required  to  take 
a WBT  is  a computer  with  a Web  browser  and  an  Internet  connection  (or  the  test  on  disk).  Test  takers  can 
take  the  WBT  whenever  and  wherever  it  is  convenient,  and  test  designers  can  share  their  test  with 
colleagues  all  over  the  world  and  receive  feedback. 

Even  though  security  is  a big  issue  in  the  use  of  WBTs,  there  are  still  advantages  to  delivering  the  test  via 
the  Web,  that  is,  no  specialized  software  necessary,  existing  facilities  like  computer  labs  can  be  used  as 
testing  centers. 

• Easy  to  write:  Whereas  producing  traditional  CBTs  requires  a high  degree  of  programming  expertise  and 
the  use  of  specially-designed  and  non-portable  delivery  platforms,  WBTs  are  comparatively  easy  to  write 
and  require  only  a free,  standard  browser  for  their  display.  In  fact,  anybody  with  a computer  and  an 
introductory  HTML  handbook  can  write  a WBT  without  too  much  effort,  and  anybody  with  a computer  and 
a browser  can  take  the  test;  language  testers  do  not  have  to  be  computer  programmers  to  write  a WBT.  This 
is  largely  due  to  HTML's  not  being  a true  programming  language  but  only  a set  of  formatting  commands, 
which  instruct  the  client's  Web  browser  how  to  display  content.  In  addition,  HTML  contains  elements  that 
support  the  construction  of  common  item  types,  such  as  radio  buttons  for  multiple-choice  items,  input 
boxes  for  short  response  items,  and  text  areas  for  extended  response  items  (essays  or  dictations). 

• Affordability:  A WBT  is  very  inexpensive  for  all  parties  concerned.  Testers  can  write  the  test  by  hand  or 
with  a free  editor  program  without  incurring  any  production  costs  except  the  time  it  takes  to  write  the  test. 
Once  a test  is  written,  it  can  be  uploaded  to  a server  provided  by  the  tester's  institution  or  to  one  of  many 
commercial  servers  that  offer  several  megabytes  of  free  web  space.  Since  WBTs  tend  to  be  small  fdes  of  no 
more  than  a few  kilobytes,  space  on  a free  server  is  usually  more  than  sufficient  for  a test.  The  use  of 
images,  sound,  or  video  can  enlarge  the  test  considerably,  however,  and  may  require  the  simultaneous  use 
of  several  servers  or  the  purchase  of  more  space. 

CATs  are  possible  on  the  Web  and  do  not  pose  many  technical  problems  beyond  those  encountered  in 
linear  tests  but  it  cannot  be  emphasized  enough  that  the  design  of  a sophisticated  WBT  is  a very  complex 
undertaking  that  requires  considerable  expertise  in  IRT. 

The  DIALANG  Project 

In  this  section  we  will  focus  on  the  use  of  WBTs  in  Second/Foreign  Language  Self-assessment,  based 
mainly  on  the  DIALANG  project,  (www.dialang.org). 

DIALANG  is  one  of  so  many  examples  of  WBTs.  It  is  a new  European  project  which  has  developed 
diagnostic  language  assessment  tools  in  fourteen  European  languages,  delivered  over  the  Internet.  The  fourteen 
languages  of  the  DIALANG  project  include  all  the  official  EU  languages  as  well  as  Irish,  Icelandic  and  Norwegian. 
The  assessment  materials  cover  all  levels  from  beginners  to  advanced;  there  are  separate  assessments  for  reading, 
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writing,  and  listening  as  well  as  for  structures  and  vocabulary.  The  project  complements  traditional  approaches  to 
language  assessment  by  exploring  the  use  of  new  technologies  for  assessing  the  use  of  language. 

DIALANG  offers  validated  tests  of  different  language  skills,  together  with  a range  of  feedback  and  expert 
advice  on  how  to  improve  the  skills.  It  also  offers  scientifically  validated  self-assessment  activities  and  allows  users 
to  determine  their  language  level,  strengths  and  weaknesses  as  well  as  to  increase  their  awareness  of  current  skills 
and  of  what  it  means  to  know  a language.  It  has  been  developed  by  more  than  twenty  major  European  institutions, 
with  the  backing  of  the  European  Commission  and  is  based  on  the  Council  of  Europe's  Common  European 
Framework  (CEF)  of  Reference  (2001),  which  has  become  established  throughout  Europe  as  the  most  widely 
recognized  frame  of  reference  in  the  field  of  language  learning. 

The  purpose  of  the  DIALANG  assessment  system  is  to  provide  language  users  and  learners  with  diagnostic 
information  about  their  language  proficiency  which  will  help  them  to  become  aware  of  their  strengths  and 
weaknesses  and  to  find  ways  of  improving  their  proficiency  (Alderson  2005).  The  assessment  system  will  also  help 
learners  locate  themselves  in  relation  to  the  levels  of  the  CEF,  for  a variety  of  uses  specified  in  the  Framework  and 
associated  documents.  Feedback  from  the  system  will  also  provide  learners  with  information  that  can  be  used  as  a 
guide  when  setting  new  learning  goals  and  planning  their  future  language  learning. 

Like  other  WBTs,  it  offers  many  advantages  such  as  the  ones  mentioned  above:  flexibility  in  time  and 
space,  easiness  to  write  and  affordability.  As  opposed  to  proficiency  tests,  the  main  advantage  that  it  offers  is  its 
security:  it  would  be  useless  to  memorize  lists  of  vocabulary,  or  to  take  the  test  several  times  in  order  to  learn  the 
specific  items,  because  the  basis  of  the  test  is  self-assessment.  The  test  does  not  provide  a certificate  to  enter  a 
university,  or  to  obtain  a better  professional  position.  Trying  to  cheat  the  machine  would  result  into  cheating  oneself. 

Furthermore,  DIALANG  offers  the  test  takers  the  possibility  to  choose  the  skill,  skill  level,  sub-skill, 
topics,  etc.  that  they  whish  to  assess.  All  this  learner  oriented  control  means  that  the  test  is  designed  for  a large 
variety  of  learners.  Any  candidates  preparing  for  any  kind  of  examinations,  following  any  kind  of  teaching 
methodology,  and  aging  16  and  over  could  benefit  from  the  test  assessment. 

The  main  gain  of  the  project  is  probably  the  way  in  which  it  provides  assessment  information: 

• The  feedback  is  immediate. 

• Scores  are  objective. 

• There  is  not  a global  grade  but  an  identification  of  the  test  taker’s  current  level. 

• It  suggests  learners’  further  study. 

• It  enables  the  planning  of  curricula. 

• It  tries  to  cover  learner’s  needs 

• It  allows  the  storage  of  the  results  for  later  comparison  to  check  progress. 

The  procedure  to  follow  in  order  to  take  the  test  does  not  require  a lot  of  computer  skills.  The  first  thing  to 
do  is  to  choose  the  language  in  which  instructions  will  be  given.  Instructions  can  be  chosen  in  a different  language 
from  the  language  tested  in  order  for  the  student  to  understand  them  whatever  his/her  level  of  proficiency  in  the 
target  language  is.  After  that,  the  skill  and  language  to  be  tested  are  chosen.  Then,  test  takers  have  the  option  to  take 
a placement  activity  to  measure  their  vocabulary  knowledge  in  the  target  language  and  determine  the  items  that 
should  be  present  in  the  assessment.  The  next  step  is  an  optional  battery  of  self-assessment  questions  that  the  learner 
has  to  answer  in  order  to  receive  some  feedback  about  what  he/she  thinks  his/her  level  is.  Next,  learners  take  the  test 
on  the  skill  chosen  previously.  The  items  are  frequently  multiple  choice  questions,  but  sometimes  a word  or  phrase 
has  to  be  written.  In  these  cases,  only  one  answer  is  possible.  Finally,  DIALANG  displays  the  results  in  terms  of 
level,  answer  verification,  score  in  the  placement  test,  self-assessment  feedback  and  advice. 
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Please  choose  the  type  of  feedback  you  would  like  to  see  You  can  return  to  tfrs  menu  at  any 


Advice 


Figure  2.  Feedback  Menu 


The  window  under  “your  level”  shows  the  level  of  the  learner’s  proficiency  according  to  the  CEF.  Under 
“advice”,  information  about  the  current  level  is  provided  such  as  the  capabilities  of  the  learner,  and 
recommendations  for  further  development  that  would  facilitate  the  step  into  the  next  level  are  granted. 

In  figure  3 it  can  be  seen  the  information  that  DIALANG  provides  about  the  test  takers’  level  of 
proficiency  indicating  the  type  of  speech  that  they  understand,  what  they  generally  understand  and  their  conditions 
and  limitations. 


4 IMnUnfc  EicgDi 


d> 

.v  b2  I 81  I D2  I C1/C2  I A1  > A2 


The  table  below  shows  some  of  the  Key  differences  between  A1  and  the  level  above  (A2): 


A1 

A2 

Wtiat  types  of 
speech  1 understand 

very  smpre  phrases  about  myself.  people 
I knew  and  things  around  me  Questions, 
instructions  and  directions  Examples 
everyday  expressions,  questions, 
instructions,  short  ano  snipe  directions 

1 &mpie  phrases  and  expressions  about 
things  important  to  me  Simple.  everyday 
corrversabons  ar>3  discussions  Everyday 
matters  in  the  media  Exampres 
messages,  routine  exchanges,  directions. 
TV  and  radio  news  terns 

What  1 understand 

Nome?  and  simple  words.  General  idea. 
Enough  to  respond  pnovidng  personal 
into,  Mowing  directions 

Common  everyday  language.  Simple, 
everyday  conversations  and  dscu wions. 
Tne  main  pont  Enough  to  follow 

Conditions  and 
limitations 

Clear,  slow  and  carefully  articulated 
speech  When  aadresseo  by  a 
sympathetic  speaker. 

Clear  and  slew  speech  Will  require  the 
hep  of  sympathetic  speakers  anci'cr 
images  Wli  sometimes  ask  for  repetition 
or  reformulation 

Figure  3.  Information  provided  about  the  test  taker’s  level  of  proficiency 
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A real  challenge  for  CBTs,  CATs  and  WBTs  is  to  assess  productive  skills:  speaking  and  writing. 
DIALANG  lacks  at  the  moment  a system  to  assess  listening  and  writing  in  terms  of  full  sentences,  paragraphs  or 
essays.  However,  a project  to  include  these  deficiencies  is  currently  taking  place  (Lancaster  University  n.d.).  These 
are  some  of  the  items  that  are  in  the  experimental  phase: 

• Indirect  speaking  with  audio  clips  as  alternatives.  After  listening  to  a recording,  the  test  taker  has  four 
options  which  are  also  recordings  intended  to  be  the  answer  or  reply  to  the  first  recording.  This  technique 
simulates  speaking  since  the  answer  is  not  written,  but  oral.  At  the  same  time  it  is  a multiple  choice  item, 
as  we  can  see  in  figure  4. 


You  are  having  problems  with  your  Cable  TV  reception  You 
telephone  the  Service  Desk  on  713  05  29  and  hear  the  following 
recorded  message: 


BOOB 


Which  of  the  following  four  messages  would  you  leave  on  the  answering  machine? 


Listen,  and  then... 

A 
B 
C 
D 


BEDE 

BEE 

BEDE 

BEE 


...  Make  your  choice! 


OVersion  A 
OVersion  B 
OVersion  C 
OVersion  D 


How  did  you  do? 


Figure  4.  Indirect  Speaking  with  Audio  clips  as  Alternatives 


• Interactive  picture  with  sound.  Oral  instructions  are  given  and  the  learner  has  to  give  an  answer  interacting 
with  the  computer.  In  figure  5,  instructions  are  given  to  locate  a room  in  a map.  The  answer  is  not  the 
traditional  written  multiple  choice  item,  but  locating  the  room  in  the  map. 


• Re-organization.  In  this  activity  some  sentences  have  to  be  rearranged  in  order  to  make  a coherent  sentence 
(figure  6).  Test  takers  have  to  drag  and  drop  the  sentences  until  they  find  the  order  they  consider  correct. 
This  technique  is  taking  us  to  a closer  step  towards  the  evaluation  of  coherence  and  cohesion  through  a 
computer. 
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Reorganise  the  sentences  in  the  paragraph  to  make  a logical  story 

Narmare  1600  gaster  sail  utanfor  monarkens  trapalats  och  frossade. 

Tack  vare  dieten  hade  kungen  blivit  av  med  en  tredjedel,  circa  70  kilo,  av  de  200  kilo 
sore  han  tidigare  vagde. 

Nar  kungen  av  Tonga  for  ett  par  dr  sedan  firade  sin  75-drsdag  ordnades  ett  stort  kalas. 
Omkring  400  grisar  slaktades  och  ons  bagare  hade  gjort  en  enorm  tdrta  till  kalaset. 
Kungen  sjalv  var  dock  forsiktig,  eftersom  han  holl  diet  sedan  ett  par  Sr. 

| Clue  | | Done  | 


Figure  6.  Re-organization 

Benchmarking  in  direct  writing  is  the  opportunity  to  assess  free  writing.  Learners  are  given  a prompt,  and 
they  have  to  write  freely  about  it.  After  they  finish  their  text,  they  are  given  a range  of  sample  texts  to 
compare  with  their  own.  Figure  7 provides  an  example.  The  sample  texts  are  answers  that  could  be  given 
by  a learner  in  a determined  CEF  level.  As  a final  assessment  tool,  advice  is  provided  stating  the  features 
that  characterize  each  kind  of  writing  level  and  some  tips  to  help  improve  writing  skills. 


Click  the  buttons  below  to 
see  the  different  sample 
texts.  These  texts  represent 
different  levels  of  writing 
skills  (A1-C2).  Compare  your 
text  to  these  samples. 

OView  answer  A1 
OView  answer  A2 
OView  answer  B1 
OView  answer  B2 
OView  answer  Cl 
OView  answer  C2 

From  the  list  below, 
choose  the  text  which  you 
feel  was  closest  to  your 
level. 


answer  A1  1^1 1 Tips  | 


a 


Figure  7.  Benchmarking  in  direct  writing 


Self-assessment  in  speaking.  A question  is  posed  for  which  several  oral  answers  are  given.  The  test  taker 
has  to  decide  whether  he/she  can  do  it  better  than  the  recordings.  Based  on  the  answer,  a level  is  assigned 
to  the  reply  and  feedback  information  about  the  level  is  provided  (figure  8). 


Speech  sample  no.  1 


® CD 

Can  you  do  it  better? 

OYes  ONo 

|~5one  | 


Figure  8.  Self-assessment  in  Speaking 

Conclusion 

The  importance  of  assessment  in  Second/Foreign  Language  learning  is  unquestionable.  The  feedback 
received  from  assessment  can  be  used  to  place  a student  in  the  correct  classroom,  to  measure  learners’  progress,  to 
revise  and  develop  curricula,  to  create  material,  to  propose  teaching  methods,  etc.  That  is  why  it  is  not  surprising 
that  so  much  attention  is  focused  on  improving  the  assessment  system.  Technology  plays  a very  important  role  in  the 
development  of  testing  methods.  There  are  different  kinds  of  computerized  assessments  such  as  CBTs,  CATs  and 
WBTs  that  offer  great  advantages  to  assessment  research:  They  provide  immediate  feedback,  ways  to  store  test 
results  for  further  analysis,  improved  security,  storage  of  large  amounts  of  items,  grading  objectivity,  multimedia 
presentations,  test  takers’  self  pacing,  etc. 

Flowever,  there  are  certain  areas  that  pose  a real  challenge  to  computer  testing.  Computers  lack  human 
intelligence  to  assess  direct  speaking  ability  or  free  written  compositions.  It  is  here  where  research  is  studying  new 
pathways  to  improve  assessment.  D1ALANG  is  a current  project  that  aims  these  goals.  DIALANG  researchers  are 
designing  innovative  test  items  to  diagnose  learners’  levels  of  proficiency,  and  the  improvements  are  having  a 
considerable  impact  on  the  considerations  of  computer  testing. 

DIALANG  focuses  on  self-assessment  taking  into  account  the  learners’  interest  in  grading  him/herself 
fairly,  but  it  could  not  be  used  as  a certification  system.  No  wonder  that  future  research  will  take  into  account  the 
progress  achieved  so  far  and  will  try  to  apply  it  to  other  kind  of  tests,  such  as  placement  tests,  certification  systems, 
etc.  Furthermore,  although  we  cannot  expect  computers  to  equal  human  intelligence,  DIALANG  is  demonstrating 
that  there  are  ways  to  improve  computer  testing.  We  will  be  expectant  to  hear  new  advances  that  will  surprise  us  as 
much  as  DIALANG  latest  findings. 
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E-Moderating  Personas:  Employing  Interaction  Design  User  Models  to 
Construct  and  Interpret  Online  Instruction  Archetypes 

Maria  D.  Avgerinou 
Carina  Andersson 

Abstract 

Phase  1 of  this  comparative  study  has  focused  on  perceptions  of  emerging  instructional  roles  and  performance  of 
teachers  operating  in  blended  (US)  and  online  (Sweden)  academic  environments.  The  current  study  (Phase  2)  has 
concentrated  on  the  creation  of  e-moderating  archetypes  (personas)  to  provide  better  understanding  of  the  roles, 
needs,  and  aspirations  of  online  teachers.  Two  e-moderating  personas  have  emerged  sharing  many  similarities  about 
their  perceptions  and  manifestations  of  their  online  teaching  styles,  which  however  are  explained  by  distinctively 
different  reasons. 


Introduction 

Web-based  distance  education  has  become  a “real”  teaching  alternative  which  has  instigated  more  research 
(The  Swedish  Net  University  Agency,  2005)  particularly  in  the  past  decade  (Avgerinou,  Griffin,  & Pettersson,  2005; 
Andersson,  Avgerinou,  Griffin,  Muffoletto,  & Pettersson,  2004;  Pettersson  & Andersson  van  Limbeek,  2002; 
Pettersson,  Svensson,  & W;crn.  2002).  This  research  has  commonly  focused  on  four  major  perspectives: 
information-technology,  course  management  systems  (including  course  communication  tools),  information  design, 
and  online  learning. 

Not  surprisingly,  there  are  many,  significant  differences  between  face-to-face  (f2f),  and  web-based 
teaching.  In  a traditional  teaching  context,  communication  is  implemented  synchronously  insofar  as  teacher  and 
students  are  able  to  communicate  instantly,  and  without  delay.  Besides  the  direct  and  instantaneous  verbal 
exchanges,  synchronous  communication  also  transpires  through  visual  signs  and  body  language. 

Conversely,  if  we  accept  that  online  teaching  (e-moderating)  occurs  via  the  Internet  and  electronic 
multimedia,  including  the  use  of  language  (both  oral  and  written),  a common  way  of  teaching  in  this  context  is 
through  a dynamic  interaction  among  participants,  yet  without  the  physical  presence  of  the  teacher  or  students.  What 
happens  then  to  the  teacher’s  “identity”  and  role  when  s/he  is  no  longer  physically  present  as  in  a traditional 
classroom?  How  is  it  established?  Perhaps  most  importantly,  how  is  it  perceived  by  both  the  students,  and  the 
teacher  him/her-self? 

Rourke,  Anderson,  Garrison,  and  Archer  (2001)  describe  a model  which  concerns  the  teaching  and  learning 
transaction  via  computer  conferencing.  The  model  presents  three  core  components  of  importance  for  learning  in  an 
online  instructional  community,  namely,  (1)  cognitive  presence,  (2)  teaching  presence,  and  (3)  social  presence. 
Teaching  presence  refers  to  “designing  and  managing  learning  sequences,  providing  subject  matter  expertise,  and 
facilitating  active  learning”  (Rourke  et  ah,  2001,  ]|  2). 

Similarly,  four  major  roles  of  the  online  instructor  have  been  identified  by  Bonk,  Kinley,  Hara,  and  Paz- 
Dennen  (2001)  -pedagogical,  social,  managerial,  and  technological.  The  pedagogical  aspect  is  manifested  through 
the  instructor’s  assuming  the  role  of  facilitator  or  moderator,  e.g.  ask  questions,  probe  responses,  encourage  student 
knowledge  building  and  linking,  summarize  or  weave  discussion,  and  support  and  direct  interactive  discussion, 
design  a variety  of  educational  experiences,  provide  feedback,  referring  to  outside  resources  and  experts  in  the  field. 
The  social  aspect  is  manifested  through  the  instructor’s  creating  a friendly  and  nurturing  environment  or  community 
feel,  exhibiting  a generally  positive  tone,  fostering  some  humor,  displaying  empathy  and  interpersonal  outreach  (e.g. 
including  welcoming  statements,  invitation,  and  apologies),  and  personalizing  with  discussion  of  one’s  own  online 
experiences.  The  managerial  aspect  is  manifested  through  the  instructor’s  coordinating  assignments  (e.g.  explain 
assignments,  set  plans  for  receipt  of  assignments,  assign  partners  and  groups,  set  due  dates  and  extensions  for 
assignments),  managing  online  discussion  forums  (e.g.  set  pace,  focus,  agenda),  and  handle  overall  course 
structuring  (e.g.  organize  meeting  times  and  places,  set  office  hours,  clarify  distributions).  The  technological  aspect 
is  manifested  through  the  instructor’s  assisting  with  user  technology  and  system  issues,  diagnosing  and  clarifying 
problems  encountered,  notifying  when  a server  is  down,  and  explaining  limitations. 
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Research  Focus  and  Questions 


This  two-phase,  comparative  research  study  was  conducted  by  two  university  instructors  whose  research 
agenda  concentrates  on  teacher  perceptions  of  own  qualities,  roles,  and  functions  in  online  instructional 
environments. 

Phase  1 of  our  research  compared  and  discussed  the  emerging  instructional  styles  of  participating  online 
teachers  (e-moderators),  in  Sweden,  and  the  USA  (Andersson  & Avgerinou,  2006).  Primary  research  questions 
concerned  how  teachers  experience  online  teaching  versus  f2f  teaching,  as  well  as  how  the  academic  environment 
affects  the  teacher  experience.  The  framework  of  this  comparative  research  was  defined  by  the  fact  that  all  US 
teachers  were  attached  to  a university  just  beginning  to  experiment  with  online  and  blended  delivery;  whereas  online 
and  blended  curricula  and  delivery  were  already  well  established  in  the  Swedish  teachers’  academic  context. 

Secondary  questions  included  the  differences  and  similarities  of  online  vs.  traditional  teaching,  and  how 
participating  teachers  come  to  recognize  and  deal  with  them.  What  kind  of  teaching  styles  are  used  in  web-based 
learning?  What  information  design  tools  may  a teacher  deploy  toward  the  creation  of  their  “identity”,  and  how  does 
s/he  establish  their  presence  online?  What  are  online  teachers’  professional  development  needs,  and  to  what  extent 
are  they  covered  by  their  respective  institutions?  Finally,  what  are  their  goals  and  desires  toward  a pedagogically 
sound,  and  effective  implementation  of  online  learning? 

If  e-moderating  is  indeed  a new  type  of  instruction  (Maor,  2003),  then  there  is  a pressing  need  to  revisit  the 
role  of  the  online  instructor.  Therefore,  in  light  of  Phase  1 research  results,  the  second  phase  (Phase  2)  of  this  study 
has  concentrated  on  using  the  identified  perceptions,  behaviors,  motivations,  and  aspirations  of  the  participating  e- 
moderators  with  the  view  to  creating  composite,  albeit  fictional  online  instruction  archetypes  (personas).  In  our 
attempt  to  do  justice  to  the  analysis  and  presentation  of  Phase  1 data,  we  set  out  to  experiment  with  the  transfer  and 
applicability  to  our  respective  blended  and  online  teaching  contexts,  of  the  persona  research  method  typically 
deployed  by  usability  and  interaction  design.  Consequently,  our  questions  have  focused  on  whether  and  to  what 
extent  this  methodological  innovation  would  produce  valid  and  reliable  results  from  a research  design  perspective; 
as  well  as,  whether  we  would  gain  a better,  more  comprehensive  and  more  realistic  insight  into  e-moderating,  as  a 
result  of  looking  at  our  existing  data  through  the  novel  lens  of  the  persona  method.  An  implication  of  the  latter 
referred  to  whether  our  e-moderating  personas  could  also  be  in  some  way  meaningful  to  all  those  involved  in  the 
various  aspects  of  online  teaching. 


Overview  of  Phase  1 


Methodology 

A predominantly  qualitative  design  was  adopted  for  this  research,  including  surveys,  semi-structured 
individual  interviews  conducted  f2f,  and  focus  groups  interviews  conducted  online.  Although  we  looked  to  address 
specific  questions  during  the  individual  and  focus  group  interviews,  there  was  still  opportunity  for  participants’ 
personal  input  which  could  be  diverting  from  the  original  set  of  questions.  The  respondents  (N=  10)  were  teachers 
currently  teaching  or  having  recently  taught  through  web-based  environments  in  blended/hybrid  (US)  and 
completely  online  (Sweden)  formats  in  our  respective  universities/countries.  Analysis  of  the  data  employed  the 
content  analysis  technique  for  all  aforementioned  instruments. 

Results 

Data  analysis  based  on  within  and  across  group  comparisons  (Andersson  & Avgerinou,  2006)  have 
indicated  some  common  threads  which  are  completely  aligned  with  the  four  major  roles  of  the  online  instructor  as 
identified  by  Bonk  et  al.  (2001). 

All  participating  teachers  exhibit  engagement  with  online  teaching  in  all  four  identified  roles.  What  is  more 
impressive,  regardless  of  their  academic  background  and  current  instructional  context,  they  not  only  view  e- 
moderating  as  a new  type  of  instruction;  but  they  also  perceive  themselves  as  constructivist  pedagogues  whose 
purpose  is  to  design  and  provide  the  type  of  nurturing  and  friendly  learning  environments  that  are  mostly  conducive 
to  student  creating  their  own  interpretations  of  reality,  and  thus  becoming  responsible  for  their  own  lifelong 
learning.  Some  of  our  teachers  share  similar  views  on  the  Socratic  philosophy  and  its  online  applications.  These 
notions,  combined  with  the  online  teacher  training  all  of  them  received  prior  to  their  involvement  in  online  teaching, 
may  perhaps  explain  their  self-perception  of  their  online  teaching  performance  from  a multitude  of  perspectives-  the 
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most  significant  one  being  those  teachers’  constant  and  fearless  quest  to  serve  their  students’  needs  in  the  best 
possible  way  while  updating  and  adapting  their  own  practice  to  the  rapidly  changing  demands  of  our  times.  As  Good 
(2001)  states,  “the  e-learning  pedagogue  is  a hybrid  creature  with  multiple  skills  and  a passion  for  learning!”  (p. 

173). 


The  Current  Study  (Phase  2) 


Research  Focus  and  Purpose 

From  a qualitative  research  methodological  standpoint,  typically  it  has  been  challenging  to  present 
qualitative  data  so  that  it  truly  reflects  meanings,  behaviors,  and  intentions  of  research  participants.  In  our  case,  it 
was  speculated  that  qualitative  data  could  be  more  realistically  and  comprehensively  reflected  via  personas,  a 
composite,  albeit  fictional  archetype  deployed  by  information  architects  and  interaction  designers  (Kuniavsky, 

2003).  Therefore,  in  our  attempt  to  do  justice  to  the  analysis  but  also  the  presentation  of  Phase  1 data,  we  set  out  to 
introduce  the  persona  research  method  to  a new  territory,  namely  our  respective  blended  and  online  learning 
contexts,  and  experiment  with  the  mechanics  of  its  transfer  and  overall  applicability. 

Taking  into  consideration  Phase  1 results,  Phase  2 of  our  research  has  focused  on  the  development  of  e- 
moderating  personas  representing  identified  behaviors,  motivations,  and  aspirations  of  real  participants  with  the 
additional  goal  to  better  understand  online  teaching  styles,  thus  assisting  the  professional  development  of  aspiring 
and  existing  online  teachers.  An  extension  of  this  goal  is  to  give  aspiring  e-moderators  a realistic  insight  into  the 
online  instructor’s  role(s),  but  also  to  assist  existing  e-moderators  better  understand  and  further  develop  their  online 
teaching  style.  Finally,  the  results  of  this  study  could  benefit  all  those  involved  in  supporting  online  teaching 
initiatives  - such  as  instructional  designers,  course  management  system  designers,  technical  staff,  professional 
development  units,  administrators,  etc.  - trying  to  effectively  match  and  align  teaching  and  learning  needs  with 
corresponding  services  and  resources. 

Consequently,  our  questions  have  focused  on  whether  and  to  what  extent  this  methodological  innovation 
would  produce  valid  and  reliable  results  from  a research  design  perspective;  as  well  as,  whether  we  would  gain  a 
better,  more  comprehensive  and  more  realistic  insight  into  e-moderating,  as  a result  of  looking  at  our  existing  data 
through  the  novel  lens  of  the  persona  method.  An  implication  of  the  latter  referred  to  whether  our  e-moderating 
personas  could  also  be  in  some  way  meaningful  to  all  those  involved  in  the  various  aspects  of  online  teaching. 

Personas-  What  are  they,  where  did  they  come  from,  and  who’s  using  them? 

User  representations  are  not  new  to  product  design,  marketing  and  branding.  Over  the  years,  information 
architects,  interaction  and  product  identity  designers  have  developed  various  models  of  user  representation  in  order 
to  base  their  product  development  on  well  informed  decisions.  Pruit  and  Adlin  indicate  (2006)  that  in  marketing  and 
branding  models  have  been  developed  by  Sissors  (1966),  Upshaw  (1995),  Weinstein  (1998),  Mello  (2002),  and 
Moore  (2002). 

In  the  area  of  usability  and  interaction  design,  scenarios  have  been  deployed  toward  the  organization, 
justification,  and  communication  of  ideas.  More  recently,  the  use  of  scenarios  has  been  extended  to  assist  in  system 
design,  product  development,  as  well  as  research  in  human/computer  interaction.  Nonetheless,  Carroll  (1995)  argues 
that  scenarios  instead  of  focusing  on  users  and  their  actual  opinions,  they  tend  to  provide  assumptions  about  them. 
This  is  apparently  the  reason  why  user-focused  models  such  as  user  profiles  (Hackos  & Redish,  1998),  user  roles 
(Constantine  & Lockwood,  2001),  as  well  as  user  archetypes  (Mikkelson  & Lee,  2000)  have  been  adopted  by  most 
usability  specialists. 

In  The  Inmates  are  Running  the  Asylum,  Alan  Cooper  (1999)  asserts  that  a focus  on  technology  is  fostered 
in  whatever  we  do,  when  we  should  instead  be  focusing  on  people.  Therefore  he  offers  a user  model  {personas ) in 
order  to  address  this  problem,  and  re-focus  product  design  and  development  on  people  and  their  needs.  Originally 
surfacing  in  1995,  Cooper’s  personas  were  used  in  a collaborative  fashion  by  design  teams  to  communicate  different 
user  perspectives  regarding  complex  design  solutions  for  consulting  projects. 

According  to  Cooper  (1999,  p.  24),  “personas  are  not  real  people  . . . they  are  hypothetical  archetypes  of 
actual  users  . . . defined  with  significant  rigor  and  precision”.  Personas  can  be  conceptualized  as  imaginary  people 
that  we  create  as  stand-ins  for  concrete  target  users  of  design  products.  When  working  with  personas  it  is  important 
to  be  aware  of  the  distinction  between  stereotypes  and  archetypes.  Each  of  these  concepts  stems  from  a specific 
theoretical  context.  Used  mostly  in  social  psychology,  stereotypes  can  be  defined  as  a simplified,  often  generalized 
form  of  idea  (National  Encyklopedin,  2006).  It  may  be  a preconceived  idea  of  the  characteristics  typifying  a person 
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or  a situation.  A stereotype  may  be  considered  as  an  attitude  based  on  such  a preconception  (Oxford  English 
Dictionary.  2006).  The  concept  of  archetype  is  based  on  Jung’s  theories.  Archetypes  represent  uniformity  and 
regularity  in  our  apprehensions  (Oxford  English  Dictionary,  2006).  Personas  are  the  official  mask,  the  face  we  show. 
In  other  words,  a persona  conveys  the  connection  between  the  ego  and  the  environment  (National  Encyklopedin, 
2006). 

The  term  “hypothetical  archetype”  denotes  Cooper’s  belief  that  there  is  no  way  to  prove  that  the  personas 
truly  are  representative  of  actual  users  until  after  the  product  is  released  and  used.  Thus,  personas  are  better 
understood  as  representing  real  people  throughout  the  design  process.  Cooper’s  point  on  defining  personas  “with 
significant  rigor  and  precision”  implies  that  the  value  of  personas  lies  in  their  ability  to  provide  specific  details  about 
users.  A very  important  element  in  Cooper’s  personal  model  is  that  the  representation  emanates  from  distinct  user 
goals,  in  addition  to  behaviors,  tasks,  demographic  information.  Personas  are  not  “made-up”.  They  are  discovered  as 
a by-product  of  the  research  process,  and  are  defined  by  their  goals.  To  make  personas  more  realistic,  names  and 
personal  details  are  constructed. 

Personas  are  a sine-qua-non  in  product  and  interaction  design,  for  various  reasons.  First,  designers  do  not 
always  design  with  the  user  in  mind.  Rather,  they  tend  to  focus  more  on  themselves  and  their  own  perspectives. 
Second,  users  are  complicated  and  varied.  It  takes  an  enormous  amount  of  effort  and  time  to  understand  their 
perspectives,  behaviors,  needs,  and  goals.  Finally,  those  who  perform  the  user  and  market  research  usually  are  not 
the  designers.  As  a result,  user  profiles  may  not  be  as  reflective  of  the  actual  user  beliefs,  needs,  and  goals. 

Gindin  (2006)  provides  psychological  evidence  as  to  the  reason  why  personas  work.  He  explains  that  when 
a designer  tries  to  anticipate  how  others  will  behave  in  a new  situation,  in  essence  he  exercises  a very  rudimentary 
skill  that  all  of  us  apply  daily  in  our  attempt  to  anticipate  how  others  may  react  to  what  we  may  say  or  do.  Grudin's 
argument  (2006,  p.  643-644)  is  based  on  three  assertions: 

(1)  We  find  natural  to  create  and  use  models  of  other  people.  If  we  did  not,  persona  construction  would 
not  make  sense.  (2)  Our  ability  to  engage  with  models  of  real  people  transfers  to  models  of  fictional 
people  (in  this  case,  personas).  (3)  Our  models  of  other  people  have  a certain  degree  of  complexity  and 
detail.  If  people  routinely  use  sophisticated  models  to  anticipate  behavior,  sophisticated  models  of 
potential  users  could  better  help  us  anticipate  their  reactions  to  design.  If  the  models  we  use  are  simple, 
expending  resources  on  detailed  or  complex  construction  efforts  might  not  be  worthwhile. 

Like  other  user  representations,  personas  have  been  used  in  various  disciplines  to  infuse  user  data  into  other 
processes.  Pruit  and  Adlin  (2006,  pp.  36-37)  assert 

Usually,  these  user  representations  are  built  and  as  static  documents  or  other  artifacts  that  provide  a 
snapshot  of  interesting  and  relevant  information  about  users.  These  artifacts  have  proven  helpful, 
largely  because  they  help  make  information  about  users  highly  accessible,  engaging,  and  memorable  to 
people  making  decisions.  These  representations  are  not  alive,  however.  They  are  depicted  as  motionless 
portraits,  usually  contained  within  a single  finite  and  static  document.  There  is  no  room  for  growth  or 
development.  That  is,  unlike  a character  in  a book  or  film,  personas  do  not  evolve.  Moreover,  the  team 
using  them  is  supposed  to  “get  to  know  them”  almost  instantly.  When  we  get  to  know  a friend, 
neighbor,  colleague,  or  even  a character  in  a favorite  book  or  TV  show,  we  build  up  an  understanding  of 
them  (i.e.,  we  develop  a relationship  with  them).  Once  we  know  people,  we  are  able  to  understand  why 
they  do  what  they  do,  what  they  want,  and  what  they  need. 

Engendering  this  level  of  understanding  is  the  next  frontier  for  user  representation.  We  believe  you  have 
to  enable  personas  to  “come  to  life,”  allowing  them  to  develop  in  the  minds  of  the  people  using  them. 

To  be  very  clear,  we  are  not  suggesting  that  personas  change  drastically  over  time,  take  on  new 
characteristics,  or  develop  new  skills  (they  are  not  to  be  moving  targets).  Instead,  we  believe  that 
personas  must  live  in  the  minds  of  your  colleagues. 

Towards  this  end,  Pruit  and  Adlin  (2006,  p.  37)  propose  that  persona  practitioners  must  (a)  embrace  the 
challenge  of  communicating  information  about  users  through  narrative  and  storytelling;  (b)  maintain  a lifecycle 
perspective  when  educating  colleagues  about  personas;  (c)  allow  the  people  using  the  personas  to  extrapolate  from 
and  extend  them. 
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The  Process  of  Persona  Construction  for  Interaction  Design 

At  first,  a persona  hypothesis  is  developed.  Based  on  likely  behavioral  differences,  while  also  taking  into 
consideration  identified  target  markets  and  demographics,  the  persona  hypothesis  attempts  to  address  such  questions 
as  what  different  sorts  of  people  might  use  a product;  how  might  their  behaviors  vary;  and,  what  ranges  of  behavior 
and  types  of  environments  need  to  be  explored.  Once  potential  roles,  behavioral,  demographic,  and  environmental 
variables  have  been  identified  for  the  persona  hypothesis,  qualitative  data  collection  begins.  This  data  is  collected 
through  ethnographic  interviews,  namely  a combination  of  immersive  observation  and  directed  interview  techniques 
(Cooper  & Reimann,  2001).  The  researcher  observes  users  in-situ  and  asks  pertinent  questions  that  primarily  focus 
on  what  motivates  user  behaviors,  and  how  they  hope  to  accomplish  their  goals.  Other  sources  of  data  collection 
may  include:  user  interviews  outside  of  their  use  contexts;  information  about  users  supplied  by  subject  experts; 
focus  groups  and  surveys;  literature  reviews  of  previous  studies. 

Once  data  collection  and  analysis  are  completed,  the  process  of  persona  construction  begins  with  a short, 
precise  narrative  description  for  each  persona.  The  description  includes  specific  details  about  workflow  and  daily 
behavioral  patterns,  professional  and  personal  needs  and  goals,  technical  skills,  as  well  as  a few  personal  details  in 
order  to  make  each  persona  unique  and  memorable.  To  ensure  development  of  a believable  archetype  a real  name 
and  a photo  is  also  included.  Depending  on  the  quantity  and  quality  of  research  data,  usually  one  to  two  primary 
personas  are  developed;  whereas  secondary  personas  may  also  emerge. 

Methodology 

For  our  Phase  2 research  design,  we  used  the  data  already  collected  and  analyzed  during  Phase  1.  Each  of 
us  was  responsible  for  the  set  of  data  that  she  had  originally  collected.  Consequently,  we  were  aiming  at  creating 
two  primary  personas,  one  for  the  US,  and  one  for  the  Swedish  participants  respectively.  We  did  not  anticipate  being 
able  to  construct  secondary  personas  due  to  the  limited  number  of  research  participants. 

We  followed  the  persona  construction  steps  as  described  above  with  the  exception  of  in-situ  participant 
observation  (study  limitation).  Taking  into  consideration  emerging  themes  and  patterns  in  participants  responses  as 
those  related  to  various  aspects  of  online  teaching,  our  narrative  description  of  each  persona  was  based  on  the 
following  categories:  Name  & Portrait  (Photo);  Personal  Description;  Technological  Skills;  Roles;  Tasks;  Current 
Problems;  Professional  Development  Needs;  Desires  and  Goals;  Values. 

Presentation  and  Discussion  of  Research  Results 

As  anticipated,  two  primary  e-moderating  personas  emerged  on  the  basis  of  participant  responses.  “Linda 
Jeffries”  represents  the  five  US-based  teachers  operating  in  blended  formats,  while  “Henrik  Johansson”  represents 
the  five  Sweden-based  teachers  responsible  for  totally  online  courses.  Both  personas  are  presented  in  Tables  1,  and  2 
below. 

Due  to  the  investigational  nature  of  this  study,  the  lack  of  previous  studies  in  this  area,  the  limited  number  of 
participants,  as  well  as  the  lack  of  in-situ  observations,  it  is  best  to  consider  these  personas  as  assumption , or 
tentative  personas.  Let  us  also  be  reminded  of  Cooper’s  point  (1999)  that  it  is  almost  impossible  to  know  how 
representative  our  personas  are  until  we  see  them  in  action.  As  explained  earlier  in  this  document,  a major  concern 
of  the  current  study  referred  to  the  transfer  and  applicability  to  a new  context  - namely,  our  respective  blended  and 
online  teaching  environments  - of  a research  method  (persona)  typically  deployed  by  usability  and  interaction 
design.  Upon  recognizing  that  analysis  and  presentation  of  qualitative  data  typically  runs  into  the  challenge  of 
finding  a manner  that  truly  reflects  meanings  and  intentions  of  research  participants,  we  searched  for  methods 
employed  by  other  disciplines  to  overcome  this  problem.  Thus  we  opted  for  persona  modeling  in  an  attempt  to 
present  and  organize  our  qualitative  data  in  brief,  yet  succinct  and  comprehensive  tables. 

From  a methodological  perspective,  and  despite  the  aforementioned  study  limitations,  we  were  particularly 
excited  to  gain  insight  into  the  potential  of  the  persona  method  toward  a more  realistic  understanding  of  the  online 
teacher’s  roles  and  needs.  In  general,  when  designing  personas  for  interactive  web  solutions  one  finds  representative 
variables  and/or  categories,  and  constructs  personas  that  are  goal-directed,  and  defined  by  goals,  tasks  or  skills  levels 
(Cooper  & Reimann,  2003).  In  this  study,  Linda  and  Henrik  were  developed  as  models  that  represent  online 
teachers’  perceptions,  behaviors,  needs  and  goals  in  relation  to  their  teaching  roles  and  perceived  styles.  Defining 
our  categories  was  a rather  uncomplicated  task  due  to  the  fact  that  we  had  already  identified  emerging  themes  and 
patterns  from  our  Phase  1 research.  Another  consideration  related  to  selecting  only  those  categories  that  while 
reducing  our  data  they  would  still  represent  it  effectively.  This  challenge  was  however  counteracted  by  the  very 
process  of  persona  creation.  We  discovered  that  persona  modeling  truly  allows  for  data  presentation  in  a narrative 
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form  so  that  the  researcher  is  able  to  gain  a holistic  view  of  the  data.  A narrative  form  implies  a beginning,  middle, 
and  end  of  a story.  It  presents  characters  and  plots.  We  came  to  realize  that  presenting  our  qualitative  data  in 
narrative  form  has  not  only  assisted  us  in  distinguishing  the  most  critical  variables/categories,  but  also  provided 
structure,  depth,  and  meaning  to  the  verbal  information  we  had  collected.  We  were  able  to  see  connections  in  our 
data  that  were  not  readily  visible  earlier.  It  is  also  noteworthy  that  while  persona  modeling  was  in  progress,  more 
than  once  we  run  the  risk  of  treating  the  data  in  an  over-simplified  or  generalized  manner.  However,  returning  to  the 
theoretical  premise  that  personas  are  archetypes  and  not  stereotypes  helped  us  avoid  that  risk.  After  all, 
generalizations  are  out  of  the  question  when  working  with  qualitative  data  at  such  a high  level  of  specificity  as  in  the 
process  of  personal  modeling.  Finally,  and  perhaps  most  importantly,  persona  modeling  forced  us  to  stay 
consistently  aware  of,  and  thus  concentrate  more  on  our  teachers’  perspectives,  and  less  on  our  own  viewpoints. 

Linda  and  Henrik  represent  in  some  sense  all  four  e-moderating  roles  identified  by  Bonk  et  al.  (2001). 
Although  both  are  particularly  concerned  with  the  pedagogical  aspect  of  their  role,  they  assign  different  weight  to 
the  other  three.  For  Henrik,  the  high  importance  ascribed  to  his  managerial  and  technical  roles,  leaves  the  social  role 
last  in  the  group.  Linda’s  second  major  focus  is  her  technical  role,  followed  by  the  managerial,  and  lastly  by  the 
social  role.  There  is  also  variation  in  their  manifestation  of  those  roles.  Both  feel  that  to  be  effective  online 
pedagogues  they  need  to  be  able  to  set  up  well-considered  online  activities,  provide  quality  resources,  quick 
responses,  frequent  group  and  individualized  feedback,  guidance,  support  and  motivation.  Anticipating  student 
needs,  as  well  as  prompting  reflective  practices  though  crucial  for  Linda,  they  do  not  feature  in  Henrik’s  list.  The 
social  aspect  of  their  role  is  mostly  manifested  through  the  online  tasks  that  they  perform  which  are  similar  in  that 
they  both  introduce  problem-based  activities,  and  a-synchronous  discussions.  However,  for  Henrik  synchronous 
online  conferences  are  important,  when  for  Linda  they  are  not  even  part  of  her  report.  Instead,  she  includes  co- 
operative learning,  reflective  activities.  The  managerial  aspect  is  more  related  to  current  problems  that  Linda 
experiences-  those  being  lack  of  effective  institutional  support,  time  management,  and  her  own  lack  of  control  over 
online  group  sizes.  Time  management  is  also  a problem  for  Henrik,  which  is  also  connected  to  his  desire  for  release 
time  toward  online  course  development.  Henrik  places  the  lack  of  effective  institutional  and  technical  support 
among  his  current  problems,  but  in  addition  he  includes  the  ambiguous  issue  of  online  security  and  its  potentially 
negative  implications  for  student  assessment.  The  latter  also  signifies  a concern  associated  with  Henrik’s  technical 
role.  He  admits  being  only  fairly  confident  in  his  online  technical  skills  despite  his  extensive  use  of  his  university’s 
course  management  system-  concerning  which,  however,  he  warns  that  it  may  be  risky  for  online  teachers  to  be 
limited  to  just  one  of  those  systems  as  this  may  inhibit  their  pedagogical  thinking  about  instructional  design 
possibilities.  Linda’s  technical  role  is  manifested  through  her  high  level  of  confidence  in  using  the  majority  of 
Blackboard’s  features,  but  also  through  her  desire  to  be  exposed  to  more  Blackboard  applications.  She  is  concerned 
about  the  fact  that  some  of  her  students  do  not  have  internet  access  which  may  affect  not  only  their  comfort  level 
online,  but  also  their  perception  of  how  seriously  they  should  take  online  learning. 

Though  both  of  them  appear  confident  in  their  online  teaching  skills,  they  still  have  specific  professional 
development  needs  and  desires.  Linda  needs  more  exposure  to  the  technical,  as  well  as  pedagogical  aspects  of  online 
teaching,  while  she  would  welcome  mentoring  if  offered  by  colleagues  with  greater  experience  than  her  in  the  area 
of  blended  learning.  Her  wish  list  includes  improvement  of  her  online  assessment  skills,  control  of  online  group  size, 
as  well  as  professional  development  opportunities  and  better  institutional  support.  Henrik’s  needs  and  relevant 
desires  focus  on  receiving  release  time  toward  online  course  development,  as  well  as  support  from  multidisciplinary 
teams  of  his  university  that  would  provide  technical,  and  pedagogical  expertise. 

Linda  and  Henrik’s  personal  descriptions  - mostly  their  demographic  and  contextual/environmental  variables 
- in  combination  with  their  individual  set  of  values  can  explain  most  of  the  similarities  and  differences  regarding 
their  aforementioned  e-moderating  profiles.  The  blended  teaching  context  does  not  create  major  problems  on  the 
technical  level  for  Linda  as  obviously  her  courses  are  not  exclusively  dependent  on  the  course  management  system. 
However,  Henrik’s  courses  being  completely  online  do  bring  the  technical  aspect  on  centre  stage  for  more  than  one 
reasons  (e.g.  access,  security,  online  testing,  synchronous  communication).  Reflection  and  co-operation  online  is 
ranked  high  in  Linda’s  teaching  agenda  due  to  their  importance  in  teacher  education  aiming  at  the  development  of 
the  reflective  educator.  However,  this  is  not  the  case  for  Henrik  whose  subject  area  is  health  care.  Linda’s  time 
management  problem  stems  from  her  juggling  her  teaching  responsibilities  between  the  f2f  and  the  online 
environment,  and  yet  she  is  very  supportive  of  the  blended  approach  due  to  the  f2f  interaction  it  affords.  In  Henrik’s 
case,  the  same  problem  can  be  explained  by  the  fact  that  he  tends  to  spend  many  hours  online  to  not  only  teach,  but 
also  to  get  to  know  his  students  better  and  thus  ‘come  closer  at  a distance’  (his  words).  It  is  due  to  his  perception  of 
the  stronger  rapport  between  teacher  and  students  also  fostered  by  the  fact  that  students  cannot  hide  online  as  they 
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can  in  £2f  teaching;  as  well  as  the  flexibility  and  creativity  of  the  online  environment  that  he  is  so  supportive  of 
online  teaching.  It  is  interesting  to  note  that  though  Linda’s  university  (small,  private)  is  only  beginning  to 
experiment  with  online  courses,  while  Henrik’s  institution  (large,  public)  is  well  established  in  the  online  teaching 
area,  they  both  - each  for  their  own  reasons  - request  better  institutional  understanding  which  would  translate  in 
more  efficient  and  effective  support.  Finally,  the  social  aspect  of  their  role  may  appear  low  in  their  agenda,  but  in 
reality  it  is  not.  Linda  tries  to  help  build  online  learning  communities  while  knowing  that  at  the  same  time  she  has 
the  luxury  and  safety  net  of  f2f  interaction.  For  Henrik,  building  rapport  online  is  also  crucial  although  he  definitely 
recognizes  that  for  this  to  happen,  absence  or  elimination  of  technical  or  managerial  problems  should  occur  first. 

Their  online  training  experiences,  but  over  and  above  their  constructivist  teaching  philosophies  can  account 
for  perceiving  e-moderating  as  a teaching  paradigm  in  its  own  right,  and  themselves  as  student-focused  online 
facilitators,  that  is,  far  from  “expert  teachers”,  just  transmitting  knowledge  to  students.  This  also  explains  why  they 
strongly  object  to  designing  courses  according  to  the  notion  of  “one  size  fits  all”.  Their  teaching  is  based  on  Socratic 
questioning,  encouraging  the  students  to  engage  in  personally  meaningful  projects,  and  find  their  own  answers  and 
solutions. 

Table  1 

Linda  Jeffries  (US  group-based  e-moderating  persona) 


Name  & & Portrait  ■ 


Linda  J effhes 


(Source:  Getty  Image  Bank) 


Personal  Description-  Linda  is  47  years  old.  She  lives  is  Chicago,  US.  She  has  been  a teacher  educator  for  over  10 
years,  having  primarily  taught  student  teachers  in  the  traditional,  face-to-face  format.  She  teaches  at  a small,  private 
university  which  is  just  beginning  to  incorporate  more  online  teaching  in  its  academic  curricula.  Linda  has  received 
training  in  Online  and  Distance  Learning  (ODL),  which  has  greatly  influenced  her  teaching  practice,  research  agenda, 
and  other  professional  activities.  In  fact  she  reports  that  her  face-to-face  and  online  teaching  styles  are  in  constant 
dialogue,  continually  influencing  each  other.  Subsequent  to  her  ODL  training,  she  has  taught  most  of  her  graduate 
courses  in  a blended  format.  She  spends  at  least  7 hours  a week  teaching  online,  logging  on  little  and  often.  She 
describes  her  online  teaching  style  as  warm,  encouraging,  reliable,  and  interactive.  Discussion  and  the  Socratic 
questioning  are  a big  part  of  it  too.  On  a personal  level,  Linda  is  married,  has  two  college  kids,  and  one  dog.  In  her 

spare  time,  she  relaxes  by  watching  movies,  and  eating  out  with  friends. 

Technological  Skills-  She  is  a great  deal  confident  in  using  her  university's  course  management  system 
(Blackboard).  The  specific  Blackboard  features  she  deploys  in  her  teaching  are  the  (1)  Discussion  Board;  (2)  Groups 
Area;  (3)  External  Links;  (4)  Course  Documents;  (5)  Course  Information;  (6)  Faculty  Information;  (7)  Assignments;  (8) 

Digital  Drop  Box;  (9)  Announcements;  and,  (10)  Online  Tests  and  Surveys. 

Online  Roles-  Reporting  a good  deal  of  confidence  in  her  online  teaching  skills,  she  normally  applies  them  through 
(1)  providing  feedback  to  the  group  of  students  on  a regular  basis;  (2)  providing  personalized  feedback  to  each  student 
on  an  as- needed  basis;  (3)  providing  guidance,  support,  encouragement,  and  motivation;  (4)  providing  quick 
responses;  (5)  settingup  online  activities;  (6)  providing  quality  resources;  (7)  stimulating  student- guided  reflections 

and  discussions;  and  (8)  anticipating  potential  student  needs  and  reporting  in  advance. 

Online  Tasks-  Typical  online  activities  that  Linda  incorporates  in  her  courses  include  (1)  General  tasks  meant  for 
everyone;  (2)  Specific  tasks  assigned  to  each  group;  (3)  Co-operative  learning  activities;  (4)  Critical  reflections;  (5) 

Problem- based  activities  (including  games  & simulations);  and,  (6)  A- synchronous  conferences 

Current  Problems-  Linda  reports  that  currently  she  struggles  with  managing  time  online.  She  doesn't  always  work 
with  the  instructionally  appropriate  student  group  size.  Her  online  assessment  skills  are  not  as  effective.  Her 
institution's  support  is  ineffective  and  inefficient.  She  worries  that  not  all  her  students  have  online  access.  Not  all  of  her 

students  are  comfortable  with  online  learning-  some  do  not  take  it  seriously. 

Professional  Development  Needs-  Linda  needs  more  hands-on  opportunities  in  order  to  experiment  with 
different  online  teaching  strategies,  but  also  learn  how  to  use  additional  Blackboard  features.  She  also  needs  a 

knowledgeable  and  compassionate  mentor  to  be  available  on  an  informal,  as- needed  basis. 

Desires  and  Goals-  Linda  would  like  to  ( 1)  incorporate  better  assessment  of  student  performance;  (2)  work  with  her 
preferred  group  size  online  (small);  (3)  receive  more  exposure  online;  (4)  receive  professional  development 

opportunities;  and,  (5)  receive  more  effective  and  efficient  institutional  support. 

Values-  Linda's  pedagogical  values  and  beliefs  regarding  online  teaching  and  learning  are  summarized  below 

■ She  believes  that  student  proj  ects  should  have  a meaning  or  value  behind  them. 

■ Although  she  believes  that  both  theory  and  practice  courses,  both  at  the  undergraduate  and  graduate  levels 
lend  themselves  to  ODL  and  Blended  learning,  she  is  more  supportive  of  blended  learning  because  it  allows 
for  face-to-face  interaction  with  students. 

_■ She  feels  that  online  teaching  involves  a set  of  skills  in  its  own  right. 
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Table  2 


Henrik  Johansson  (Swedish  group-based  e-moderating  persona ) 


1. 

Name  & Portrait- 

Henrik  J ohansson 

A 

(courtesy  of  Lasse) 

2. 

Personal  Description-  Henrik  is  44  years  old  and  lives  in  Vasteras,  one  of  the  major  cities  in  Sweden.  He  works 
at  a public  university,  and  travels  with  train  to  campus.  Although  before  he  begun  teaching  at  the  university  - 
about  6- 7 years  ago  - Henrik  did  not  receive  any  teacher  training,  he  has  since  participated  in  various  professional 
development  sessions  on  online  teaching  and  learning  provided  in-house.  He  has  a master’s  degree  as  well  as 
considerable  professional  experience  in  his  subject  area  (health  care).  Initially,  he  taught  in  the  traditional  f2f 
format.  Today  some  of  his  classes  are  f2f  and  some  are  totally  online.  He  has  approximately  four  years  of 
experience  in  online  teaching.  He  puts  in  4- 10  online  teaching  hours  per  week,  logging  on  little  and  often.  Henrik 
does  not  want  to  be  the  so  called  traditional  teacher  transferring  knowledge  to  students,  and  neither  does  he  want 
to  take  the  role  of  the  expert.  Instead,  he  strives  to  be  a good  pedagogue,  who  is  more  of  a supervisor,  giving 
guidance  and  advice.  Henrik  does  not  like  spoon-feeding  the  "correct"  answers  to  students;  instead  he  inspires 
students  to  find  their  own  solutions.  Henrik  is  married  and  lives  with  his  wife  and  two  children.  In  their  spare 
time,  he  and  his  family  spend  time  at  their  summer  house.  Henrik  has  a passion  for  designs  from  the  1950’s. 

3- 

Technological  Skills-  Henrik  is  fairly  confident  in  using  the  university's  online  course  management  system. 
Most  of  the  time,  the  specific  features  he  deploys  in  his  teaching  are  the  (1)  Discussion  Board;  (2)  Course 
Documents;  (3)  Course  Information;  (4)  Assignments;  (5)  Announcements;  and,  (6)  Faculty  Information. 

4- 

Online  Roles-  Reporting  a good  deal  of  confidence  in  his  online  teaching  skills,  he  normally  applies  through  ( 1) 
providing  feedback  to  the  group  of  students  on  a regular  basis;  (2)  providing  personalized  feedback  to  each 
student;  (3)  providing  guidance,  support,  encouragement,  and  motivation  more  than  sufficient;  (4)  providing 
guality  resources;  (5)  providing  guick  responses;  (6)  setting  up  online  activities. 

5- 

Online  Tasks-  Typical  online  activities  that  Henrik  incorporates  in  his  courses  include  (1)  General  tasks  meant 
for  everyone;  (2)  Specific  tasks  assigned  to  each  group;  (3)  Problem-based  activities  (including  games  & 
simulations);  (4)  A-synchronous  conferences;  and,  (5)  Synchronous  chat  seminars. 

6. 

Current  Problems-  Henrik  reports  that  he  has  mixed  feelings  about  the  teaching  habits  he  developed.  Online 
teaching  liberates  him  from  certain  time  and  location  constraints  associated  with  traditional  teaching.  On  the 
other  hand,  he  finds  himself  constantly  working  before  a computer,  both  at  the  university  and  at  home.  He  also 
admits  that  there  may  be  a problem  with  the  security  level,  since  we  can  never  be  sure  of  who  actually  performed 
the  tasks.  Some  students  tend  to  perform  their  online  studies  "on  the  side"  of  other  studies  or  work.  Since  we  do 
not  meet  the  students  in  real  f2f  classrooms,  we  do  not  know  how  they  function  socially.  Henrik  also  reports  a lack 
of  understanding  from  his  department  and  IT-personnel  regarding  the  quality  and  quantity  of  support  needed 
toward  effective  online  teaching. 

7- 

Professional  Development  Needs-  Henrik  reports  that  he  needs  increased  collaboration  with  IT-technicians 
and  designers  working  in  multidisciplinary  teams.  He  desires  more  effective  and  efficient  IT- support.  He  would 
also  welcome  additional  course  development  time,  since  developing  online  courses  requires  a huge  amount  of 
time,  probably  more  so  than  traditional  courses. 

8. 

Desires  and  Goals-  Henrik  would  like  to  (1)  achieve  an  increased  collaboration  with  IT-technicians  and 
designers  in  multidisciplinary  teams;  (2)  be  given  additional  working  hours  to  develop  his  courses  both  technically 
and  pedagogically;  (3)  receive  more  effective  and  efficient  IT-support. 

9- 

Values-  Henrik's  values  and  beliefs  on  online  teaching  are  summarized  below 

■ Feels  that  online  teaching  involves  a set  of  skills  in  its  own  right. 

■ Feels  "closer  at  a distance"  compared  to  traditional  classroom  teaching  as  he  gets  to  know  the  students  more 
as  individuals. 

■ Believes  that  students  cannot  "hide"  as  in  the  f2f  format. 

■ Enj  oys  the  flexibility  and  creativity  the  online  environment  affords. 

■ Thinks  it  is  risky  to  only  use  one  course  management  system  (e.g.  WebCT,  First  Class,  Blackboard  etc.)  as 
this  may  limit  creative  pedagogical  thinking. 

Recommendations 

Those  interested  in  exploring  further  this  methodological  innovation  using  our  study  as  their  starting  point, 
should  appreciate  the  need  for  further  validation  of  our  persona  hypothesis  through  surveys,  interviews,  as  well  as 
in-situ  observation  of  a larger  population  of  teachers  operating  in  both  blended  and  online  environments. 

Our  underlying  goals  were  to  give  aspiring  e-moderators  a realistic  insight  into  the  online  instructor’s 
role(s),  but  also  to  assist  existing  e-moderators  better  understand  and  further  develop  their  online  teaching  style.  We 
also  envisaged  that  the  results  of  this  study  would  benefit  all  those  involved  in  supporting  online  teaching  initiatives 
- such  as  instructional  designers,  course  management  system  designers,  technical  staff,  professional  development 
units,  administrators,  etc.  - trying  to  effectively  match  and  align  teaching  and  learning  needs  with  corresponding 
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services  and  resources.  It  is  our  sincere  hope  that  the  aforementioned  groups  will  look  carefully  into  the  potential 
that  persona  modeling  holds  toward  increasing  the  understanding  of  what  is  important  to  online  teachers,  how  they 
implement  their  teaching  online,  as  well  as  what  particular  concerns  and  needs  they  tend  to  have.  In  that  case,  as 
indeed  was  the  case  for  our  study,  personas  can  be  employed  as  a reference  point  and  a discussion  tool  for  design 
and  administration  teams,  while  at  the  same  time  assist  those  teams  making  better  decisions  and  supporting  online 
teaching  initiatives  having  people,  their  stories  and  contexts  - not  technology  - in  mind. 
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Abstract 

This  paper  addresses  the  questions:  What  is  video  podcasting?;  When  is  video  podcasting  a helpful  addition 
to  a live  or  online  course?;  What  educational  institutions  are  currently  making  use  of  video  podcasting  for 
instruction?;  and.  How  does  one  create  and  distribute  video  podcast  content?  An  overview  is  presented  of  what 
video  podcasting  is,  who  is  making  use  of  video  podcasting  for  instruction,  and  the  technical  aspects  of  creating 
video  podcast  content. 


Podcasting 

A new  technology  currently  receiving  a great  deal  of  attention  is  “podcasting.”  A podcast  is  an  audio  or 
video  fde  placed  on  the  Web  for  individuals  to  subscribe  and  listen  to  primarily  on  MP3  players  such  as  the  Apple 
iPod.  The  term  “podcast”  like  that  of  “radio”  or  “video”  can  refer  to  either  the  content  or  the  method  of  delivery.  A 
number  of  institutions  have  been  experimenting  with  providing  audio  podcasts  as  supplemental  instructional  media 
(Descy,  2005).  Winer  (2004)  observes  that  podcasting  works  similarly  to  how  a desktop  aggregator  works.  One 
subscribes  to  a set  of  feeds,  and  then  can  easily  view  the  new  items  from  all  of  the  feeds  together,  or  each  feed 
separately.  Podcasting  works  the  same  way,  with  one  exception.  Instead  of  reading  the  new  content  on  a computer 
screen,  one  listens  to  or  views  the  new  content  on  an  iPod  or  any  device  that  can  play  MP3  and/or  MPEG4  files. 

iPod  Devices  on  Campus 

Although  it  is  possible  to  listen  to  or  view  podcast  content  via  a computer,  it  is  the  near-ubiquitous  use  of 
iPods  and  similar  devices  that  have  captured  the  public’s  attention  and  the  imaginations  of  educators.  Duke 
University,  for  example,  currently  issues  incoming  freshmen  with  an  iPod  (Armstrong  Moore,  2005).  In  Armstrong 
Moore’s  article,  Greg  Joswiak,  Apple’s  vice  president  of  iPod  product  marketing  is  quoted  as  saying,  “Institutions 
such  as  Duke,  Georgia  College  & State  University,  and  others  are  using  the  iPod  as  a portable  learning  tool  for 
listening  to  recorded  lectures,  foreign  language  study,  research  notes,  storing  files  and  photos,  and  listening  to  audio 
books  and  podcasts.”  Modern  iPods  have  large  storage  capacities.  The  popular  video  iPods,  with  either  a 30  or  a 60- 
gigabyte  capacity,  allow  users  to  store  and  play  digital  video  and  still  images  as  well  as  sound  files.  The  authors  note 
from  personal  experience  that  the  video  iPod’s  2.5”  screen  is  remarkably  clear. 

One  of  the  appeals  of  the  iPod  is  that  users  have  access  to  digital  media  when  they  are  not  sitting  at 
traditional  desktop  or  laptop  computers.  Johnson  (2005)  points  out  that  portable  media  players  like  Apple’s  iPod  are 
“. . .not  home  computers  but  body  computers,  fashion  accessories  that  now  want  to  be  cameras,  TVs  and  radios.  The 
iPod  has  become  a sex  symbol  of  self-expression,  a hi-tech  fetish  that's  helped  us  see  the  media  as  something  to  be 
individually  programmed.”  There  is  certainly  an  important  novelty  aspect  to  these  portable  media  players,  but  the 
longer  term  implications  of  ubiquitous  media  access  suggests  a profound  shift  in  how  educators  might  make  use  of 
instructional  media.  Professors,  teachers,  and  instructional  designers  may  now  consider  how  to  create  or  distribute 
course  content  that  individuals  can  access  almost  any  time  or  any  place. 

Video  Podcasting  or  Vodcasting 

Although  audio  podcasting  is  a relatively  recent  technology  itself,  even  more  recent  is  the  phenomenon  of 
video  podcasting  (that  is,  podcasts  that  contain  visual  information  either  in  the  form  of  still  images,  animation,  or 
video).  Video  podcasting  or  “vodcasting”  is  seen  by  many  as  a potentially  powerful  instructional  medium  (Johnson, 
2005;  Mackey,  2005;  Warlick,  (?)).  Mackey  (2005)  suggests  that  video  podcasting’s  revolutionary  potential  is 
based  on  the  fact  that  they  can  be  made  and  distributed  via  the  Web  by  a variety  of  producers,  ranging  from  big- 
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budget  to  almost  no-budget  situations.  Podcast  video  does  not  require  special  server  distribution  software  the  way 
Internet  broadcasting  does  (SCVI.NET,  2006).  This  means  video  podcasting  may  be  useful  as  both  a means  to 
communicate  content  (Descy,  2005;  Ellis  and  Cohen,  2001;  Touvinen,  2000)  and  as  a means  of  student  media 
production  (Anderson,  2005). 


Using  Vodcasting  to  Enhance  Instruction 

Vodcasting  is  a potentially  significant  enhancement  to  instruction.  It  may  allow  students  an  opportunity  to 
receive  supplemental  multimodal  presentation,  which  is  generally  accepted  as  beneficial,  particularly  in  the 
apprehension  of  complex  concepts  (Tuovinen,  2000).  Ellis  and  Cohn  (2001)  caution  that  instructional  multimedia 
supplements  are  intuitively  attractive,  but  their  use  must  be  weighed  against  the  cost  of  production.  Vodcasting 
presents  interesting  opportunities  for  design  and  research  in  that  vodcast  productions  may  be  accomplished  with  a 
minimal  budget  and  moderate  technological  expertise. 

Educational  Institutions  Currently  Making  Use  of  Podcasts 

Various  educational  institutions  are  making  use  of  podcasting.  Major  universities  such  as  Duke,  Michigan, 
Purdue,  Stanford,  and  UC  Berkeley  are  making  podcasts  of  course  lectures  available  for  their  students  - especially 
undergraduates.  Students  enrolled  in  various  courses  use  the  Web  (typically  a password  protected  university  server) 
to  access  the  podcasts.  Purdue  University,  with  their  Boilercast  project,  is  at  the  forefront  of  using  podcasting  for 
educational  purposes.  The  idea  behind  Boilercast  is  to  provide  students  with  audio  recordings  of  lectures  allowing 
students  to  review  daily  course  materials  for  use  on  homework  or  as  a review  for  examinations.  A course  that  is  part 
of  the  Boilercast  project  will  have  a podcast  available  to  students  the  same  day  a class  lecture  is  given.  Students  at 
Purdue  University  have  access  to  podcasts  for  70  different  courses  offered  on  campus.  This  number  is  expected  to 
significantly  grow  (Purdue  University,  2005). 

Educational  Vodcasts  and  Vodcast  Design 

Perusing  the  educational  vodcasts  available  to  the  general  public,  using  primarily  the  Education  Podcast 
Network  (http://epnweb.org/)  and  Apple’s  iTunes  Music  Store,  it  becomes  readily  apparent  that  educational  video 
podcasting  is  in  its  infancy.  The  vast  majority  of  video  podcasts  are  essentially  collections  of  clips  produced 
originally  for  television  viewing.  This  is  unsurprising;  historically  new  media  begin  by  making  what  use  they  can  of 
the  content  developed  for  older  media  (e.g.  early  television  relied  heavily  on  content  “imported”  from  live  theater 
and  radio  drama).  There  are  few  examples  to  date  of  vodcasting  designed  to  make  best  use  of  this  new  medium.  The 
significant  differences  between  vodcasting  and  television  viewing  are  the  vodcast  player’s  portability  and  the  size  of 
its  view  screen.  As  the  medium  matures,  the  design  of  the  presentation  will  no  doubt  take  into  consideration  its 
unique  features.  Just  as  television  matured  into  half-hour  and  one-hour  programming  that  makes  use  of  multiple 
camera  angles  and  close-up  shots,  and  takes  into  consideration  the  fact  that  most  viewers  will  be  watching  alone  or 
in  a small  group,  which  is  very  different  from  a two  or  three-hour  live  play  or  musical  experience  which  the  viewer 
shares  with  a large  group  and  sees  from  a single  angle,  vodcast  producers  must  learn  what  works  best  for  this  new 
medium.  Some  of  the  more  sophisticated  vodcast  producers  add  subtitles  to  the  video  and  present  the  video  in 
multiple,  short  clips  to  take  better  advantage  of  the  medium.  An  example  of  which  is  PBS.org’s  NeRDTV  vodcasts 
(PBS,  2005),  the  content  of  which  is  from  PBS’s  broadcast  television  show  of  the  same  name  (see  Figure  1). 
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Figure  1.  A still  from  the  NeRDTV  vodcast  interview  with  Doug  Englebart  (2005) 


Vodcasting  Basics:  Production  Elements  and  Support  Technology 

From  the  perspective  of  accomplished  media  producers,  video  podcast  production  is  reasonably  simple 
using  multi-platform  software  such  as  Flash,  PowerPoint,  and  QuickTime.  Video  podcast  production  is  particularly 
easy  even  for  relatively  unsophisticated  media  producers  using  Apple’s  multimedia  production  software  such  as 
GarageBand,  iPhoto,  and  iMovie  FID.  Production  is  a matter  of  putting  together  a multimedia  presentation  saved  in 
.m4a,  .mov,  .mp4,  ,m4v  file  format.  In  experimenting  with  the  production  and  distribution  of  video  podcasts,  the 
authors  found  that  in  practice  the  only  reliable  method  of  creating  an  iPod-compatible  video  file  was  to  use  Apple’s 
QuickTime  software  (using  the  “Export”  function  and  selecting  “movie  to  iPod”)  or  to  use  Apple’s  GarageBand 
software,  selecting  “New  Podcast  Episode”  when  creating  the  file.  The  authors  experimented  with  creating  and 
exporting  files  from  Flash  and  PowerPoint:  both  of  these  software  programs  allow  one  to  export  a digital  video  file, 
but  these  files  ultimately  had  to  be  opened  in  and  exported  appropriately  from  either  QuickTime  or  GarageBand  to 
be  viewable  on  a video  iPod. 

Figure  2 (below)  is  a screen  capture  of  Apple’s  GarageBand  software  in  “New  Podcast  Episode”  mode. 
Note  the  tracks  that  are  typical  of  most  sound  production  software  include  a “Video  Track”  (the  topmost  track) 
which  accommodates  either  video  imported  from  iMovie  or  still  images  imported  from  iPhoto. 


34 


Figure  2.  A screen  capture  of  Apple’s  GarageBand  in  “New  Podcast  Episode”  mode. 


Once  the  video  file  is  created,  two  more  steps  are  necessary  for  the  vodcast  to  be  viewable  to  the  public 
through  Apple’s  iTunes  software  (presently  the  most  popular  method  of  collecting  and  viewing  podcasts).  The  first 
is  to  upload  the  video  file  to  a server.  Write  down  the  location  of  the  video  file  because  one  will  need  this 
information  as  one  completes  the  second  step  - creating  an  XML  file.  The  XML  file  allows  users  to  subscribe  to  the 
vodcasts  through  iTunes.  This  file  needs  to  be  uploaded  to  the  same  directory  as  the  video  file.  Figure  3 (below)  is  a 
screen  capture  of  XML  code  that  the  authors  used. 
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<?xm l vers i on= "1 . 9 " encod i ng= " UTF-8 " ? > 

<rss  xm l ns : i tunes=  "http:  //www . i tunes . com  /bTDs  /Rodeos  t- 1 . 0 . d td  " vers  i on=  " 2 . 0 " > 

<channe l > 

<t i t le>AECT  Presentation:  Dallas  2006</title> 

<i tunes: author >Rbbie  Brown  and  Tim  Green </i  tunes :author> 

< l i nk>http : //www  .geoc i t i es .com/bookwormsandgeeks/vodcastexamp le .xm l </l i nk> 

<descr ipt i on>Examples  of  Vodcasts  for  Presen tat i on</descr i pt i on> 

<i tunes :subti tle>Dig  this!  It  works!  </i tunes :subt i t le> 

<i tunes : summary >Two  examples  of  vodcasts  for  the  flECT  presentation  in  Dallas  2006.  Created  by  flbbie  Brown 
and  Tim  Green.  What  wonderful  product  quality.  I think  these  two  guys  are  probably  wonderful,  funny,  and  great  to 
hang  out  wi th.</i tunes :summary> 

< l anguage  >EN  < / 1 anguage  > 

<copyr ight>(c)  2006  flbbie  Brown  and  Tim  Green < /copy r i ght> 

<i tunes: owner > 

<i tunes: name >flbbie  Brown  and  Tim  Green</i tunes :name> 

< i tunes :ema i l >bookwormsandgeeks6yahoo .com</i tunes :ema i l > 

</i tunes :owner> 

<category>Techno logy </category> 

< i tunes : c a tegory  tex  t= " T echno l ogy " ></ i tunes : ca tegory > 

<i tem> 

<ti tle>Episode  2</title> 

<i tunes: author >flbbie  Brown  and  Tim  Green </i tunes :author> 

<descr i p t i on  > AECT  Presen  ta  t i on  < /descr i p t i on  > 

<i tunes :subti t le>Vodcasts</i tunes :subti tle> 

<i tunes: summary >fln  example  of  a vodcast  presentat i on</i  tunes : summary > 

<enc  l osur e ur l = " h t tp : // www . geoc i t i es . com /book wor msandgeeks /ep i sode02 . mo v " l eng  th= " 0029 " ty pe= " v i deo /mo v " 

/> 

<gu i d>http : //www .geoc i t i es .com/bookwormsandgeeks/ep i sode02 .mov</gu i d> 

<pubDate>Fr i , 22  Sept  2006  2:00  PST</pubDate> 

<i tunes:expl ici t>no</i tunes:expl ici t> 

<i tunes :durat ion >00 :00:29</i tunes : dura t i on > 

< i tunes : keywords >weather,  satte l i tes</i  tunes :keywords> 

</i tem> 


</channel> 
</rss>  | 


Figure  3:  A screen  shot  of  XML  code 
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There  are  several  places  in  the  XML  code  where  information  about  the  video  file  needs  to  be  inserted.  This 
information  is  unique  to  each  vodcast.  As  other  vodcasts  are  created,  additional  code  needs  to  be  added  to  the  XML 
file.  The  code  that  is  added  is  from  the  item  tag  to  the  /item  tag.  Figure  4 (below)  shows  an  example  of  additional 
code  included  for  a second  vodcast. 
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<?xm l vers i on= "1 . 0 " encod i ng= " UTF-8 " ? > 

<r ss  xm  l ns : i tunes=  "http:  //mm  . i tunes . com  /bTDs  /Podcas  t- 1 . 0 . d td  " vers  i on=  " 2 . 0 " > 

<channe l > 

<title>RECT  Presentation:  Dallas  2©06</title> 

<i tunes: author >flbbie  Brown  and  Tim  Green</i tunes : author > 

< l i nk>http : //www .geoc i t i es .com /bookwormsandgeeks/vodcastexamp le .xm  l</l ink> 

<descr i pt i on>Examp les  of  Vodcasts  for  Presentat i on</descr i pt i on> 

< i tunes :subt i t le>D ig  this!  It  works!  </i tunes :subt i t le> 

< i tunes : summary >Two  examples  of  vodcasts  for  the  flECT  presentation  in  Dallas  2006.  Created  by  flbbie  Brown 
and  Tim  Green.  What  wonderful  product  quality.  I think  these  two  guys  are  probably  wonderful,  funny,  and  great  to 
hang  out  wi th.</i tunes :summary> 

< language>EM</language> 

<copyr ight>(c)  2006  flbbie  Brown  and  Tim  Green</copyr i ght> 

<i tunes : owner > 

<i tunes: name >flbbie  Brown  and  Tim  Green </i tunes : name > 

< i tunes : ema i l >bookwormsandgeeks@yahoo . com  < / i tunes : ema i l > 

</i tunes: owner > 

<ca  tegory  >T  echno l ogy  < /ca  tegory  > 

<i tunes :category  text="Techno logy" ></i tunes : category > 

<i tem> 

<ti t le>Ep isode  2</title> 

<i tunes: author >flbbie  Brown  and  Tim  Green</i tunes :author> 

<descr i pt i on>AECT  Presentat i on</descr i pt i on> 

<i tunes  :subti t le>Vodcasts</i  tunes :subti tle> 

< i tunes : summary >fln  example  of  a vodcast  presentat i on</i tunes :summary> 

<enclosure  ur l="http: //www. geoc i t i es .com/bookwormsandgeeks/ep j sode02 .mov"  length="0029"  type="v i deo/mov" 

/> 

<gu i d >h  t tp : //www . geoc i t i es . com /bookwor msandgeeks /ep i sode02 . mov  < /gu i d > 

<pubDate>Fr i , 22  Sept  2006  2:00  PST</pubDate> 

< i tunes :exp l i c i t>no</i tunes :exp l i c i t> 

<i tunes :durat ion >00 :00:29</i tunes : dura t i on > 

< i tunes : keywords  >wea  ther , sa  t te l i tes  < / i tunes : keywords  > 

</i  tem> 

<i tem> 

<ti tle>Ep isode  1 </t  i t le> 

<i tunes: author >flbbie  Brown  and  Tim  Green</i tunes : author > 

<descr i p t i on > AECT  Pr esen  ta  t i on  < /descr i p t i on  > 

< i tunes  :subt i t le>Vodcasts</i  tunes :subt i t le>  | 

<i tunes: summary >fln  example  of  a vodcast  presentat i on </i tunes: summary > 

<enc l osur e ur l = " h t tp : //ww w . geoc i t i es . com /book wor msandgeeks /ep i sode0 1 . mov " l eng th= " 0044 " ty pe= "video /mov " 

/> 

<gu i d>http : / /www .geoc i t i es .com/bookwormsandgeeks/ep i sode01  .mov</gu i d> 

<pubDate>Fr i , 22  Sept  2006  2:05  PST</pubOate> 

< i tunes :exp l i c i t>no</i  tunes :exp l i c i t> 

< i tunes :durat i on >00 :00 : 44  </i tunes :durat i on> 

< i tunes : keywords  >wea  ther , sa  t te l i tes  </ i tunes : keywords  > 

</i  tem> 

</channel> 

</rss> 


Figure  4:  Screen  shot  of  XML  code  that  includes  two  vodcasts 


Testing  the  vodcast  is  accomplished  through  iTunes.  One  opens  iTunes  and  selects  the  Advance  drop-down 
menu  - selecting  Subscribe  to  Podcast  (this  process  is  referred  to  as  “subscribing”).  In  the  dialogue  box,  the  URL 
for  the  vodcast  will  be  entered.  As  an  example,  the  URL  created  in  the  XML  code  in  figures  3 and  4 is 
http://www.geocities.com/bookwormsandgeeks/vodcastexample.xml . Completing  this  process  will  allow  the 
vodcasts  to  be  downloaded  to  a computer  and  viewable  in  iTunes.  The  iTunes  software  will  then  also  check  to  see  if 
new  vodcasts  are  available  at  this  location  and  offer  the  user  an  option  to  download  the  new  vodcasts  as  they 
become  available. 


Discussion  and  Conclusion 

Like  film  and  television  at  their  inceptions,  video  podcasting  is  an  exciting  technology  that  has  captured  the 
imaginations  of  instructors,  students,  and  the  news  media.  There  seem  to  be  a number  of  possibilities  for  its  use  in 
instruction  and  many  individuals,  institutions,  and  organizations  are  currently  experimenting  with  producing  and 
distributing  video  podcasts.  However,  like  film  and  television,  these  instructional  possibilities  may  only  be  realized 
by  those  educators  who  put  significant  time  and  effort  into  experimenting  with  design  and  production. 

The  authors  found  video  podcast  production  software  relatively  easy  to  use.  However,  the  entire  process  of 
production  and  distribution  requires  a good  deal  of  arcane  technical  knowledge.  The  authors  spent  well  over  two 
days’  time  creating  and  making  available  two  30-second  vodcast  episodes.  Most  of  that  time  was  spent  learning  the 
processes  necessary  to  create  and  distribute  the  files;  the  initial  vodcast  episode  took  far  longer  to  create  than  the 
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second  episode.  The  authors  are  relatively  sophisticated  in  the  creation  of  digital  multimedia  and  in  the  application 
of  HTML  and  XML,  but  at  times  they  had  to  turn  to  even  more  expert  producers  to  understand  how  to  make  the 
process  of  video  podcasting  work  correctly.  The  effort  involved  was  a far  cry  from  Apple’s  claim,  “Podcasts  can 
easily  be  created  on  your  Macintosh,”  (apple.com,  2006).  It  should  be  noted  that  both  authors  used  Macintosh 
computers  to  accomplish  the  video  podcasting  experiment.  It  should  be  further  noted  that  the  experiment  was  aimed 
only  at  accomplishing  the  technical  task  of  creating  and  distributing  video  podcast  episodes;  the  episodes  themselves 
are  “content  free.”  There  was  no  time  spent  creating  actual  instructional  content,  which  would  have  added  a number 
of  extra  hours  to  the  entire  process. 

Similar  to  the  production  of  film,  traditional  video,  or  Web  sites,  most  educators  are  going  to  need  the 
support  of  knowledgeable  media  producers  and  programmers  to  make  video  podcasting  work  for  them  and  their 
students.  Educators  and  sophisticated  media  producers  and  programmers  are  also  going  to  have  to  be  able  to 
communicate  effectively  among  themselves  in  order  to  create  something  that  is  both  of  genuine  educational  value 
and  is  technologically  viable.  The  communication  between  such  disparate  groups  presents  a unique  challenge  in  and 
of  itself. 

Of  course,  in  terms  of  content  selection  and  presentation  design,  the  old  computer  programmers’  term, 
GIGO  (“Garbage  In:  Garbage  Out”)  applies  to  video  podcasting  as  much  as  it  does  to  any  instructional  media 
production  endeavor.  The  use  of  a new  technology  may  briefly  imbue  content  with  a certain  temporary  glamour,  but 
ultimately  that  content  and  its  delivery  must  be  of  some  genuine  help  to  the  learner  or  it  will  ultimately  be  discarded. 
While  it  is  an  exciting,  promising  new  technology,  it  remains  to  be  seen  what  significant  effect  on  instruction  video 
podcasting  will  have  on  the  design  of  instruction  in  the  long  run. 
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Use  of  Primary  Source  Documents  for  Teaching 
21st  Century  Information  Literacy  Skills 


Carol  A.  Brown,  EdD 
East  Carolina  University 

Introduction 

Information  rich,  information  overload,  and  information  explosion  are  popular  terms  in  our  culture  today. 
Classroom  teachers,  media,  and  technology  specialists  recognize  that  skilled  consumption  of  information  requires 
careful  selection  and  evaluation  of  resources  before  information  can  be  used  for  students’  research  projects  (Lamb 
2004).  Often  it  is  an  overwhelming  task  sifting  through  the  electronic  sources  to  locate  appropriate  and  usable 
information.  In  working  with  elementary  and  secondary  school  students,  one  of  the  most  common  roadblocks  to  the 
use  of  digital  sources  is  the  indiscriminant  copying  and  pasting  of  information  to  a student’s  research  paper. 

Teachers  are  distressed  when  students  copy  and  paste  directly  from  the  Internet  to  “write”  their  final  paper.  The 
media  or  technology  specialist  is  equally  frustrated  when  students  wander  aimlessly  browsing  the  web,  looking  for 
“interesting”  facts  about  their  topic.  More  often  than  not,  ample  time  is  devoted  to  searching  for  facts  but  students 
lack  a focused  research  question  (McKenzie,  online).  In  a typical  classroom  activity,  the  teacher  provides  a list  of 
topics  for  research  and  the  media  specialist  a list  of  possible  resources  in  the  form  of  a pathfinder  (Bush,  2003).  The 
final  paper  is  a compilation  of  information,  reporting  interesting  facts  and  figures  related  to  the  topic.  However,  even 
with  careful  planning,  many  teachers  experience  less  than  satisfactory  results  from  their  students.  Research  papers 
requiring  library  and  Internet  resources  for  research  projects  are  often  superficial  in  content  and  lacking  valid 
conclusions  statements.  The  solution  to  these  problems  may  be  in  the  use  of  a new  curriculum  that  focuses  on  “21st 
Century  Literacy”  for  K12  classrooms. 


Review  of  the  Literature. 

In  21st  century  classrooms,  the  current  trend  for  teaching  information  literacy  is  in  conjunction  with 
technology-based  tools  using  Internet  and  other  digital  information  resources  (National  Higher  Education 
Consortium,  2005;  Lamb,  2001;  2004).  Information  Communication  and  Technology  (ICT)  is  one  of  several 
components  in  the  21st  Century  Skills  Model  recommended  by  a leading  advocacy  organization  for  infusing  21st 
century  skills  into  education  (See  www.21stcenturyskills.org).  Using  ICT,  students  learn  not  only  how  to  query  a 
database  but  also  layout  the  design  of  an  electronic  slide  presentation.  In  an  article  in  Change  magazine  it  was 
reported  that  technology-savvy  students  of  the  21st  century  use  fewer  critical  thinking  skills  than  their  parents 
(Breivik,  2005).  Even  with  the  use  of  Internet,  cell  phones,  DVD’s,  and  a multitude  of  electronic  devices, 
information  comes  cheap  and  often  is  not  used  for  critical  analysis  and  evaluation.  Quick-fix  research  using  search 
engines  may  yield  copy  and  paste  solutions,  but  extended  thinking  is  often  lacking  in  the  research  process.  Even 
though  Information  Communication  and  Technology  skills  are  being  strongly  promoted  as  a critical  need  for  21st 
century  classrooms,  many  students  enter  the  workplace  with  inadequate  skills  for  use  of  information  to  solve  typical 
work-related  tasks  (Educational  Testing  Service,  2005).  Defining  the  information  need  and  articulating  a concise 
and  logical  report  is  in  high  demand  for  most  positions  in  business  and  industry  (NCREL,  2003). 

The  21st  Century  Partnership  was  established  by  the  U.S.  Department  of  Education  and  several  member 
organizations  including  the  Educational  Testing  Service  (ETS),  American  Association  of  School  Librarians  (AASL), 
and  the  International  Society  for  Technology  in  Education  (ISTE).  In  addition,  there  is  an  extensive  list  of  members 
from  the  technology  industry  (See  http://www.21stcenturyskills.org)  The  21st  Century  Learning  Model  has  six 
components  - Life  Skills,  21st  Century  Content,  Core  Subjects,  Learning  & Thinking  Skills,  21s  Century 
Assessments,  and  ICT  Literacy.  Based  on  a report  published  by  the  Educational  Testing  Service  (2005),  components 
for  ICT  Literacy  include  several  processes  in  which  the  student  must  engage  in  order  to  perform  at  the  level 
recommended  by  the  21st  century  literacy  consortium.  Students  must  define  the  information  need;  collect  and 
manage  information  from  digital  environments;  interpret  information  using  ICT  tools  for  comparison,  analysis,  and 
synthesis;  evaluate  information  for  authority,  bias,  and  timeliness;  and  communicate  their  findings  through  the 
creative  use  of  ICT  environments.  If  students  are  to  engage  in  classroom  activities  that  will  transfer  to  future  work 
environments,  there  must  be  a 21st  century  approach  to  the  use  of  information  to  solve  problems  and  report 
solutions. 
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In  elementary,  secondary,  and  college  classrooms,  skills  for  formulating  a clear  and  concise  research 
question  are  directly  related  to  Information  Skills  curriculum  recommended  by  the  American  Library  Association 
and  the  Association  for  Educational  Communications  & Technology  (ALA  & AECT,  1998,  p.  1 1).  If  you  combine 
this  with  the  ICT  Literacy  model  recommended  by  ETS,  the  first  step  would  be  to  “identify  and  appropriately 
represent  an  information  need”  (p.  18  ETS,  2005).  This  involves  behaviors  related  to  formulating  a research 
statement  and  retrieving  information  from  a variety  of  sources.  But,  this  is  just  the  first  step.  Using  the  information 
to  efficiently  generate  a summary  report  or  solutions  to  a problem  should  be  the  culminating  step  for  a real-world 
research  assignment.  This  demands  a level  of  thinking  and  content  analysis  that  is  often  missing  in  K12  research 
projects.  One  reason  might  be  an  over  reliance  on  simplistic  research  projects  using  general  information  sources 
(Loertscher,  2005).  Students  may  benefit  from  more  problem  solving  and  higher  level  learning  activities  within  their 
research  projects.  Early  in  2004  I was  invited  to  participate  in  a formative  evaluation  project  for  a primary  sources 
website.  As  a professor  of  media  and  technology  the  idea  of  using  digital  primary  source  documents  became  an 
intriguing  possibility.  I decided  to  investigate  the  effects  of  ICT  environments  for  teaching  the  use  of  primary 
source  documents  for  high  school  research  projects. 

In  2004,  the  academic  library  at  East  Carolina  University  received  a large  grant  from  the  North  Carolina 
State  Library  (North  Carolina  State  Library,  online).  Previously,  Joyner  Libraiy  had  worked  with  local  museums  to 
archive  images  of  artifacts  from  the  Country  Doctor  Museum,  and  the  Tobacco  Farm  Life  Museum.  The  purpose  of 
the  project  was  to  digitize  approximately  200  texts  pertaining  to  the  history  of  29  eastern  North  Carolina  counties. 
Currently  the  Digital  Library’  of  Eastern  North  Carolina  is  a product  of  these  early  efforts  to  preserve  old  and  rare 
books,  documents,  and  images  through  the  grant-funded  digitization  project  (available  online 
http://digital.lib.ecu.edu/histoiyfiction/  ).  The  Digital  Library’  of  Eastern  North  Carolina  follows  the  example  of 
other  online  resources  providing  digital  primary  source  documents.  The  American  Memory’  Project,  sponsored  by 
the  Library  of  Congress  is  “a  digital  record  of  American  history  and  creativity" 

(http://memory.loc.gov/ammem/about/index.html).  It  was  first  begun  in  the  early  1990's  and  currently  stores  more 
than  9 million  items  related  to  American  history  and  culture. 

In  addition  to  the  websites  sponsored  by  East  Carolina  and  the  Libraiy  of  Congress,  there  are  numerous 
Internet  resources  providing  access  to  primary  source  documents.  Formats  for  the  primary  sources  include  letters, 
diaries,  legal  documents,  maps,  broadsides,  and  many  other  resources  scanned  and  archived  as  digital  primary 
sources  (Brown,  2005;  Veccia,  2004).  Because  these  documents  are  primarily  written  as  first  person  accounts  by 
individuals,  and  are  not  “textbook  facts”,  students  soon  leam  that  careful  evaluation  of  all  sources  is  an  important 
step  in  the  research  process.  Biases  and  opinions  are  deeply  rooted  within  most  letters  and  diaries  (Friedman,  2005; 
Library  of  Congress,  online).  Even  official  documents  and  legal  reports  can  be  biased,  depending  on  geographic 
region  and  time  period.  Extensive  reading  in  any  of  the  primary  sources  leads  to  the  possibility  of  writing  your  own 
version  of  history.  Much  depends  on  how  well  the  author  presents  various  viewpoints  and  perspectives  in  the 
reporting  a singular  event.  Based  on  earlier  work  by  Friedman,  I proposed  a partnership  project  with  a rural  high 
school  in  eastern  North  Carolina  to  investigate  students'  proficiencies  with  ICT  literacy  using  electronic  resources 
and  primary  source  documents.  Using  three  Internet  based  resources  for  primary  source  documents,  the  question  was 
asked,  “are  digital  primary  source  documents  useful  tools  for  teaching  Information  and  Communication  Technology 
(ICT)  literacy  skills  in  K12  schools?” 


Method. 

In  case  studies  reported  by  Barton  (2005),  high  school  students  participated  in  perspective-taking  activities  to  write 
alternate  points  of  view  for  events  in  history.  Using  old  diaries,  letters,  and  journals,  students  soon  learned  that 
history  could  change  based  on  such  things  as  geographic  location  and  social  position  of  the  person  relating  their 
story.  Replicating  many  of  Barton’s  activities,  a case  study  investigation  was  used  to  report  the  strategies  students 
used  to  analyze  primary  source  documents  for  “writing  their  own  history”.  In  addition,  surveys  and  focus  group 
interviews  were  used  to  gather  feedback  from  students  and  their  response  to  use  of  primary  source  documents  as 
reference  materials  for  doing  research. 

A rubric  was  used  to  identify  proficiencies  in  the  seven  areas  recommended  by  Educational  Testing  Service 
for  ICT  Literacy.  These  are-  define,  access,  manage,  integrate,  evaluate,  create  and  communicate.  My  observations 
were  focused  on  these  proficiencies  and  how  often  students  demonstrate  skills  for  defining  a question,  accessing  and 
managing  information,  integrating  resources  into  useful  summaries,  evaluating  their  sources,  and  creating  a valid 
conclusion  statement  presented  electronically  to  their  peers  (ETS,  2003).  Upon  recommendation  by  the  school 
media  specialist  and  a classroom  teacher,  the  12th  grade  English  Literature  students  were  given  instruction  in  how  to 
use  (1)  skills  in  generating  a research  question,  (2)  use  specific  search  strategies  to  locate  primary  source  documents. 
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(3)  evaluate  information  for  bias  and  engage  in  perspective-taking  thinking  processes,  and  (4)  apply  technology 
literacy  skills  for  use  of  PowerPoint  software  for  the  purpose  of  reporting  conclusions  to  a peer-audience.  The 
researcher,  media  specialist,  and  classroom  teacher  provided  additional  remediation  and  support  as  students  learned 
new  skills  in  use  of  Boolean  methods  for  entering  keywords  in  a variety  of  search  engines  and  educational 
databases.  They  also  received  instruction  in  how  to  present  their  final  report  to  their  peers  in  the  classroom  setting. 

In  an  earlier  publication,  the  Educational  Testing  Service  (2003)  reported  testing  the  seven  ICT 
proficiencies  for  college  level  students,  however,  in  a recent  report  by  MSNBC  (available  online),  the  college  level 
skills  have  been  translated  for  use  in  testing  high  school  students: 

Students  will  receive  an  individual  score  on  a point  scale  of  400  to  700,  and  schools  will  get 
reports  showing  how  students  fare  in  seven  core  skills:  defining,  accessing,  managing,  integrating, 
evaluating,  creating  and  communicating  information. 

Using  the  ICT  proficiencies  as  a framework,  a rubric  was  designed  to  identify  the  seven  core  proficiencies  within 
students’  research  behaviors  and  in  the  content  of  their  final  products  (See  Appendix). 

The  first  proficiency  is  identifying  need  and  generating  the  research  question  (Define).  Students  should  be 
able  to  “frame”  (McKenzie,  online)  a research  question  using  problem-based  learning  techniques  for  brainstorming, 
perspective-taking,  and  critical  thinking  skills.  Second,  students  should  use  ICT  tools  to  gather  information  and, 
based  on  information  need  for  the  defined  research  question,  organize  the  information  into  a useful  format  (Access 
and  Manage).  Next,  students  engage  in  higher  order  thinking  skills  to  compare  and  contrast,  synthesize,  and 
summarize  information  (Integrate).  Following  integration,  students  should  be  able  to  make  judgments  on  whether 
the  information  satisfied  needs  for  the  task  in  the  ICT  environment,  including  authority  of  the  source,  bias  and 
timeliness  of  the  materials  (Evaluate).  Last,  using  ICT  tools  students  must  be  able  to  report  their  findings, 
conclusions,  and  solutions  within  appropriate  context  for  the  audience  (Create/Communicate).  The  observation 
instillment  included  the  following  behaviors:  (1)  students  ability  to  frame  their  research  questions  related  to  the  time 
period  and  circumstances  for  the  research  project;  (2)  students  ability  to  generate  a focused  research  question  that  is 
useful  to  meet  an  authentic  information  need;  (3)  students  persistence  in  use  of  search  engines  to  find  useful 
information;  (4)  students  persistence  to  work  with  software  for  developing  their  electronic  slide  presentations;  (5) 
students  problem  solving  ability  within  the  ICT  environment. 

Discussion 


Framing  an  Authentic  Research  Question. 

Students  needed  guidance  and  instruction  in  how  to  frame  and  generate  a research  question.  Even  when 
provided  a list  of  motivational  topics,  the  researcher,  teacher,  and  media  specialist  devoted  several  class  periods 
modeling  the  cognitive  process  for  “framing  the  question”  and  bringing  the  topic  to  a focused  research  question 
rather  than  a fact-gathering-and-report  methodology.  When  generating  their  research  questions,  typical  responses  by 
the  students  might  be  - "research  on  battles  of  the  Civil  War",  or  "research  on  life  on  a southern  plantation".  Students 
were  not  able  to  frame  their  topics  within  the  boundary  of  a focused  research  question.  Nor  were  they  able  to 
generate  an  original  query  that  would  require  critical  analysis  on  information  related  to  a topic.  Instead,  students 
wrote  superficial  questions  that  only  required  searching  for  basic  facts.  In  Illustration  1 you  will  see  two  examples  of 
focused  questions  generated  by  one  male  and  one  female  from  the  class.  The  students  gathered  pertinent  information 
and  critically  analyzed  the  content  for  perspectives  based  on  culture,  geography,  and  personal  bias.  By  framing  the 
research  questions,  students  were  able  to  define  the  information  needs,  access  appropriate  sources  of  information, 
and  evaluate  the  content  of  information  for  usefulness  in  answering  their  research  questions.  Note,  all  three 
proficiencies,  define,  access,  and  evaluate  are  components  for  the  ICT  skills  curriculum. 
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See  Illustration  1.  PowerPoint®  slides  displaying  students’  “framed”  research  question. 


Framing  Questions 

What  was  prison  life  like  at  different 
prisons  during  the  Civil  War? 

What  was  the  difference  between  the 
treatment  at  Union  Prisons  and 
Confederate  Prisons? 


They  purpose/ 

We/  often/  forget  how  difficult  Ct 
way  for  women/ to  wiev  the/ 
buttle/  for  the/  vote/. 

What  did/ they  sacrifice/? 

Why  would/  someone/  oppose/ 
women/  y suffrage/? 


Many  students  needed  additional  instruction  in  how  to  evaluate  the  contents  and  validity  of  certain  websites.  In 
particular,  credentials  of  the  author,  was  an  important  information  skill  learned  by  students  in  this  case  study. 
Proficiencies  for  ICT  Literacy  include  the  ability  to  "judge  the  degree  to  which  information  satisfied  the  needs  of  the 
task  in  ICT  environments,  including  determining  authority,  bias  and  timeliness  of  materials."  In  Illustrations  2 and 
3,  are  examples  of  how  students  gathered  information  from  several  sources  for  critical  analysis.  The  students’  final 
conclusions  are  based  on  various  perspectives.  Considerable  judgment  and  evaluation  of  content  was  needed  for 
students  to  report  on  final  conclusions  related  to  their  original  research  questions. 

Illustration  2.  An  example  of  student’s  ability  to  use  ICT  tools  to  Access,  Integrate,  Create,  and  Communicate 
information  needed  to  answer  a framed  research  question. 


NARRATIVE  OF  PRISON  LIFE 
AT  BALTIMORE  AND  JOHNSON'S  ISLAND,  OHIO: 

By:  Shepherd  Henry  E.  Elli 

• Building  was 
uncleansed  and  not 
ceiled.  It  was  simply 
weather  boarded. 

• Rations:  One  half  loaf 
of  hard  bread  and  a 
piece  of  salted  pork. 

• Clothing  and  blankets 
were  not  suitable  for 
weather  conditions. 

http://docsouth.unc.edu/shehperd/shepherd.html 
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Similarities  and  Differences 


Union  Prisons 

• Small  portions  of  food 

• Clothing  was  not 
suited  for  the  weather 

• Received  cruel  and 
harsh  punishment 

• More  medicine 
available. 


Confederate  Prisons 

• Smaller  portions  of 
food  than  received 
than  in  the  Union 
Prisons 

• Clothing  was  not 
suited  for  weather 

• Received  cruel  and 
unusual  punishment 

• No  medicine  available 


Illustration  2 is  an  example  of  how  students  gathered  information  from  several  sources  with  first  hand  reports  on 
prison  conditions  in  the  North  compared  with  conditions  as  reported  in  the  South  during  the  American  Civil  War. 
The  student’s  conclusions  are  based  on  analysis  of  facts  as  these  were  reported  by  persons  incarcerated  in  both 
northern  and  southern  military  prison  campus. 

In  Illustration  3,  the  student  reported  the  comments  from  the  opposition  as  a strategy  to  report  how  bias  is 
evident  in  the  media.  She  compares  this  with  media  reports  from  broadsides  that  were  in  support  of  women  voters. 
The  student’s  conclusion  contained  some  personal  bias  but  was  still  well  articulated  in  her  admiration  for  early 
leaders  in  the  suffrage  movement. 

Illustration  3.  An  example  of  student’s  ability  to  use  ICT  tools  to  synthesize,  summarize,  compare  and  contrast 
information. 


The  Women’s  Reasons 

“ Women  must  obey  the 
laws... pay  taxes...  suffer 
from  bad  government...  just 
as  men  do. 

Women  are  citizens  of  a 
government  of  the  people, 
by  the  people  and  for  the 
people,  and  women  are 
people. 

They  should  vote  equally 
with  men.” 


VOTES  FOR  WOMEN! 

THE  WOMAN'S  REASON. 

BECAUSE 

MW  H w fa- 

“'•‘’■"■■‘Sir— • — — — - 



***  ——  T .. 

WHY? 

BECAUSE 

*—  owfa  T.CIW1WH* 

NjbMrf  Airvn 

m Womai  StArifc  Aitoorton 
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The  Opposition 

•The  majority  of  women 
were  believed  to  not 
want  the  vote. 

•Another  argument  was 
that  the  responsibilities 
of  voting  would  interfere 
with  their  duties  as  a 
housewife. 

•Women  were  also 
expected  to  trust  men 
with  their  interests. 


DO  WOMEN  WANT  THE  VOTE? 


Would  H Promote  (he  Ceneral  Welfare  ? 


Persistence  in  Use  of  Search  Strategies. 

Early  in  the  project,  students  wasted  too  much  time  with  aimless  wandering  of  the  Internet  to  locate  “facts 
about  my  research”.  This  relates  to  the  typical  practice  of  locating  basic  facts  without  using  higher-level  thinking 
for  comparing  and  contrasting,  summarizing,  and  critical  analysis  of  what  was  reported  in  the  documents.  Students 
also  struggled  with  fact  versus  opinion  with  many  of  the  documents.  Legal  documents,  government  websites,  and 
other  historical  documents  presented  in  textbook  style  were  always  interpreted  and  reported  as  entirely  accurate. 
Students  soon  learned  that  documents  were  not  always  entirely  accurate.  They  were  prompted  to  compare  several 
different  viewpoints  for  one  historical  event  and  soon  recognized  the  differences  in  geographic  region,  time  period, 
and  economic  status  can  affect  the  accuracy  of  content  within  primary  documents.  Cognitive  processes  for  this 
activity  clearly  support  the  ICT  skill  for  integrate  in  which  the  student  must  "interpret  and  represent  information" 
thinking  skills  to  "synthesize,  summarize,  compare  and  contrast  information  from  multiple  sources"  (ETS  2003). 

Persistence  in  Use  of  Software  for  Problem  Solving  and  Presentation  of  Research  Results. 

Students  were  motivated  to  report  their  findings  using  electronic  slide  presentations  however  many  students 
were  frustrated  in  learning  basic  computer  skills  and  trouble-shooting  for  glitches  in  the  software.  There  was  also  a 
need  for  prompting  in  the  use  of  search  engines  and  Boolean  methods  for  entering  keywords.  The  earlier  pilot 
project  was  successful  in  that  students  were  highly  motivated  to  research  their  topic  and  because  of  the  opportunity 
for  additional  time  in  the  library  using  the  Internet.  However,  the  teacher  reported  less  than  satisfactory  results  for 
the  final  product.  Student  papers  did  not  include  valid  conclusion  statements.  When  interviewed  during  focus 
groups,  students  reported  lack  of  time  to  write  the  final  paper.  With  additional  probing  and  questioning,  the 
researcher  concluded  that  students  spent  too  much  time  with  Internet  searching  for  the  primary  sources.  During  the 
follow-up  projects,  (through  collaborative  efforts  of  the  teacher  and  media  specialist)  an  extended  timeline  was 
planned  to  provide  adequate  time  for  gathering  data,  processing  the  information,  and  writing  final  conclusions.  In 
addition,  students  were  provided  a list  of  sources  in  a pathfinder  and  were  instructed  to  search  extensively  within  the 
websites  and  web  portal  provided  by  the  media  specialist.  This  helped  alleviated  the  problems  with  too  much  time 
searching  irrelevant  websites. 


Conclusions 

Students  gained  new  understanding  in  how  to  interpret  historical  documents  as  they  read  primary  sources  to 
“write  their  own  history”.  They  also  learned  that  all  history  is  a biased  report  of  events  and  are  the  viewpoint  of  the 
author.  Comparison  of  viewpoints  by  age,  race,  gender,  socio-economic  status,  and  geographic  region  were  used  to 
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teach  information  literacy  for  the  selection  and  evaluation  of  information  to  answer  their  self-generated  research 
questions.  Students  began  with  simple  fact-gathering  but  soon  graduated  to  more  in-depth  analyses  in  order  to 
identify  biased  and  opinioned  reports  for  an  historical  event.  In  addition,  students  exercised  skills  in  evaluation  of 
several  different  sources  for  the  same  event.  Their  evaluations  were  self-directional  and  required  a high  level  of 
evaluative  judgment  for  drawing  final  conclusions.  This  was  an  obstacle  for  many  of  the  students  early  in  the  project 
until  they  recognized  the  importance  for  comparing  stories  from  different  perspectives.  The  American  Civil  War, 
Women’s  Rights,  and  sensitive  topics  such  as  slavery  of  African  Americans  were  useful  topics  leading  to  a 
successful  experience  with  information  and  communication  technology  skills.  Using  the  seven  proficiencies 
recommended  by  the  Educational  Testing  Service  (ETS,  2003,  2005),  I was  able  to  devise  an  observation  instrument 
with  criteria  for  students’  ability  to  locate  and  access  information.  Using  recommendation  by  ETS,  a researcher- 
designed  rubric  was  used  for  assessing  skills  in  determining  usefulness  of  information.  In  addition,  criteria  within 
the  rubric  were  used  to  evaluate  conclusions  presented  in  students’  electronic  PowerPoint®  slides.  In  the  beginning 
frustration  was  high,  but  mastery  of  basic  technology  skills  led  to  a more  relaxed  learning  environment.  Just  as  with 
adult  learners,  high  school  students  are  pressured  to  work  under  time  constraints  when  facing  technology  glitches 
and  slowdowns.  The  teacher  and  the  media  specialist  were  highly  motivated  to  achieve  a successful  completion  for 
the  projects,  thus  persevered  even  with  technology  delays.  The  effective  use  of  Information  and  Communication 
Technology  skills  as  described  for  21st  Century  classrooms  are  evident  from  this  study. 
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Appendix 

Rubric  for  Evaluating  Contents  of  Research  Projects  * 


*adapted  from  Educational  Testing  Service  Components  of  ICT Literacy’. 


Defme-Using  ICT  tools: 

Select  appropriate  topic  - teacher  input 

Frame  research  question  within  focused  boundaries 
Identify  appropriate  resources  to  answer  the  research 
question 

Access-Using  ICT  tools: 

Locate,  retrieve  digital  primary  sources  useful  for 
answering  the  research  question 

Manage-  Using  ICT  tools: 

Organize  information  to  be  able  to  efficiently  analyze 
content 

Summarize  content 

Report  portions  of  content  to  answer  the  research 
question 

Integrate-  Using  ICT  tools: 

Read  and  interpret  information  using  synthesis, 
summarization,  and  critical  thinking  skills  for  compare, 
contrast  and  perspective  taking. 

Read  and  interpret  information  from  multiple  sources 

Evaluate-  Within  ICT  environments: 

Make  judgments  on  the  usability  of  the  information  to 
answer  the  research  question 

Make  judgments  on  authority  of  the  source 
Make  judgments  on  bias 

Make  judgments  on  timeliness  of  the  materials  as  this 
relates  to  the  research  question 

Create-  Using  ICT  tools: 

Adapt,  apply,  and  design  a report  with  conclusions  to 
the  original  research  question 

Communicate-  Using  ICT  tools: 

Design  a report  that  is  appropriate  for  the  audience 
Design  a report  with  content  that  is  clearly 
communicated  and  understood  by  the  audience. 
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Developing  Learning  Content  Markup  Language  (LCML)  for  Personalized 
Instruction  in  the  Adaptive  Learning  System 
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University  of  Memphis 

Abstract 

This  paper  designs  a Learning  Content  Markup  Language  (LCML)  based  on  XML  structure.  LCML 
focuses  on  developing  effective  structure  of  learning  objects  called  learning  content  for  the  adaptive  learning  system, 
Matching  Learning  Objects  to  Individual  Differences  (MLOID)  proposed  at  the  AECT  conference  in  2005.  Learning 
content  includes  descriptive  metadata,  which  is  pedagogical  information  and  properties  related  to  individual 
differences,  and  links  to  material  source.  The  learning  content  written  by  LCML  will  be  a basis  of  complete  learning 
units  in  the  adaptive  learning  system. 


Introduction 

Adaptive  learning  systems  aim  to  help  learners  gain  their  own  knowledge  structure  by  personalizing 
instruction  based  on  individual  differences  (Baek,  Wang  & Lee,  2002;  Cheon  & Grant,  2005).  The  key  function  of 
an  adaptive  learning  system  is  determining  how  to  modify  the  learning  content  according  to  learners’  differences.  To 
deliver  appropriate  personalized  instruction,  an  instructional  unit  should  be  subdivided  into  learning  objects  (Cheon 
& Grant,  2005).  The  concept  of  learning  objects  has  been  given  immense  attention  from  scholars,  and 
standardization  efforts  have  been  conducted  by  IEEE  Learning  Technology  Standard  Committee  (LTSC)  and 
Instructional  Management  System  (IMS)  Global  Learning  Consortium.  However,  most  standardization  with  e- 
learning  is  concerned  with  issues  of  interoperability  and  reusability  for  learning  objects  and  content  packaging 
(Hummel,  Manderveld,  Tattersall  & Koper,  2004).  Reusing  learning  objects  without  considering  instructional 
principles  may  lead  e-learning  to  create  “page-turners,”  which  do  little  to  advance  interactions  and  apply  appropriate 
instructional  models. 

In  order  to  implement  effective  adaptive  learning  system,  new  concept  of  learning  objects  is  needed. 
Separating  structure  and  contents  from  layout  could  make  it  possible  to  create  and  update  contents  for  different 
purposes  (Fiala,  Hinz,  Meissner  & Wehner,  2003;  Mohan  & Greer,  2003).  Extensible  Markup  Language  (XML)  is 
suitable  for  implementing  the  separation.  Because  XML  is  independent  from  a specific  platform  and  emphasizes 
document  structure,  it  allows  multi-format  output.  The  focus  of  this  paper  is  to  design  Learning  Content  Markup 
Language  (LCML)  based  in  XML  in  order  to  implement  Matching  Learning  Objects  to  Individual  Differences 
(MLOID)  proposed  at  AECT  conference  2005. 

Review  of  relevant  literature 

In  order  for  learning  objects  to  be  reusable  and  searchable  in  a consistent  way  across  different  platforms, 
they  can  be  simplified  by  providing  standardized  information  called  metadata  (Mohan  & Greer,  2003).  There  have 
been  a number  of  efforts  worldwide  to  develop  standard  for  learning  objects.  For  example,  IEEE  Standard  for 
Learning  Object  Metadata  (LOM)  is  the  first  accredited  standard.  The  LOM  standard  uses  nine  categories  of  XML 
data  elements  to  describe  learning  objects.  It  is  intended  that  LOM  metadata  will  simplify  the  discovery, 
management,  and  exchange  for  learning  objects  over  the  Web  (Mohan  & Brooks,  2003).  The  other  standard  model 
is  IMS  Metadata  Specification  (IMS,  2003).  It  has  specifications  for  the  design  of  learning.  IMS  content  package 
specification  is  the  structure  of  a learning  object  and  physical  resources,  and  IMS  learning  design  specification 
consists  of  contents  plus  learning  activities  and  achievement  of  learning  objectives  (Mohan,  2004). 

However,  there  have  been  arguments  about  the  specifications  (e.g.,  Hummel  et  al..,  2004;  Mohan,  2004; 
Mohan  & Brooks,  2003).  Hummel  et  al.  insisted  that  technical  standards  are  important  but  not  enough  to  promote 
learning  without  attention  to  the  process  of  instruction.  On  the  other  hand,  Mohan  (2004)  stated  that  even  if  LOM 
standard  contains  elements  such  as  Semantic  Density  (i.e.,  rated  effectiveness)  and  interactivity  level,  it  is  not 
enough  to  facilitate  sophisticated  instructional  design.  In  addition,  pedagogical  information  should  be  included  in 
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metadata  for  personalization.  Pedagogical  information  would  specify  cognitive  activities  plus  teaching  and  learning 
strategies  (Mohan  & Brooks,  2003). 

Most  studies  about  learning  objects  have  used  or  suggested  new  XML -based  document  formats  for 
adaptive  learning.  One  of  the  studies  proposed  Domain  Structure  Markup  Language  (DMML)  which  aimed  to 
describe  the  knowledge  general  structure  (Baek  et  al.,  2002),  where  elements  such  as  title,  concept,  relationships, 
explanations  and  examples  are  used  to  create  pages  dynamically.  TeachML  uses  didactical  units  and  didactical 
structures  to  create  different  presentations  and  routes  through  the  course  (Wehner  & Lorz,  2001).  In  a component- 
based  document,  components  are  described  by  attaching  metadata  specifying  properties  and  adaptive  behavior, 
constructing  a Web  site  by  aggregating  and  linking  components  to  complex  document  structures  (Fiala  et  al.,  2003). 

Prototype  of  Learning  Content  Markup  Language  (LCML) 

The  proposed  prototype  of  Learning  Content  Markup  Language  (LCML)  focuses  on  developing  effective 
structure  of  learning  objects  for  the  adaptive  learning  system  called  MLOID.  Regarding  previous  studies,  there  are 
limitations  for  implementing  an  effective  adaptive  learning  system  by  only  using  metadata  and  learning  objects 
alone.  In  order  to  fully  realize  adaptive  learning  units  for  each  individual,  content  should  be  separated  from 
presentation.  In  addition,  pedagogical  strategies  should  be  considered  when  constructing  the  learning  unit. 

The  concept  of  learning  objects  in  the  system  is  not  the  smallest  media  file  but  contains  necessary 
information  such  as  structure  or  level  of  difficulty  as  well  as  the  link  of  the  learning  objects,  such  as  a text  file  or  an 
image  file.  Accordingly,  learning  contents  include  descriptive  metadata  and  material  sources  as  a basis  for  complete 
learning  units  in  the  adaptive  learning  system.  However,  metadata  outside  of  learning  content  will  exist  in  order  to 
be  searchable  and  reusable.  The  basic  architecture  of  adaptive  learning  contents  is  depicted  in  Figure  1. 
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[Figure  1]  The  learning  content  adaptation  model 


Learning  content  is  expressed  by  XML  grammar,  because  XML  is  independent  from  presentation  and 
easily  extended  by  alternative  elements.  Therefore,  learning  content  written  by  LCML  is  system  independent, 
reusable  and  extensible.  Elements  in  XML  documents  describe  properties  for  adaptation.  The  structure  and  example 
of  XML  tags  in  LCML  are  described  in  Table  1 . 
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[Table  1]  LCML  structure  example 


Elements 

Attribute  examples 

Use  for  adaptation 

Topic 

Title 

Title  of  a page 

Section 

Type  - pretest,  instruction,  assessment 
Title 

Navigation,  menu  bar 

Instructional  model 

Name  - Gagne’s  nine  events  of  instruction 
Type  - linear,  nonlinear 
Step  name  - present  stimulus  materials 
Step  number  - 2 

Instruction  route 

Content 

Type  - pretest,  presentation,  activity 

Prior  knowledge 

Type  - general,  high,  low 

Different  contents 

Learning  style 

Type  - visual,  auditory,  reading 

Different  presentation 

Media 

Type  - text,  image,  sound,  movie,  PPT,  activity, 

Learning  materials 

Using  appropriate  attributes  of  each  element  in  learning  contents,  appropriate  adaptive  learning  unit  can  be 
reproduced.  For  example,  the  <topic>  and  <section>  tags  can  be  used  to  make  titles  or  navigation  menus  in  different 
interface  styles.  Furthermore,  the  -^instructional  model>  element  consists  of  four  attributes  that  indicate  pedagogical 
information.  Especially  the  step  number  will  be  used  to  construct  sequences  of  the  learning  contents.  In  the  <content 
type>  element,  a learning  content  may  not  have  any  learning  material  but  have  guidance  of  a learning  activity. 

<Prior  knowledge>  and  <learning  style>  are  basic  indicators  for  personalization  in  this  system.  Depending  on  the 
types  of  those  elements,  an  adaptation  model  will  make  personalized  instruction.  Because  of  the  flexibility  of  XML, 
additional  individual  differences  elements  can  be  added. 

Future  work 

In  this  paper,  LCML  is  proposed  as  the  first  step  to  implement  the  proposed  adaptive  learning  system 
MLOID.  Learning  content  written  by  LCML  can  fully  realize  effective  adaptive  learning  system.  Separating 
contents,  including  properties  of  instructional  information  and  individual  differences,  allows  reproducing  a unit  of 
learning  in  order  to  meet  various  adaptive  purposes.  However,  there  are  considerations  in  implementing  learning 
contents  using  LCML.  First,  additional  XML  elements  for  other  individual  differences  components  could  be 
considerable.  Second,  an  XML  authoring  tool  needs  to  be  developed  in  order  for  instructional  designers  to  easily 
make  learning  contents  and  assign  the  XML  elements.  The  adaptive  system  can  make  various  personalized  learning 
only  based  on  the  essential  learning  contents.  Third,  a student  profde  XML  schema  needs  to  be  designed  to 
accumulate  a student’s  learning  outcomes  and  preferences.  Last,  a server-side  adaptation  algorithm  based  on 
learning  contents  model  is  needed. 
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Teacher  Created,  Computer  Based  Virtual  Field  Trips 
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Abstract:  This  paper  will  discuss  Computer  Based  Virtual  Field  Trips  that  use  technologies  commonly  found  in  any 
public  school  classroom  in  the  United  States.  The  discussion  will  focus  on  the  advantages  of  both  using  and  creating 
these  field  trips  for  an  instructional  situation.  If  the  decision  is  made  by  an  instructor  to  create  a virtual  field  trip,  the 
technologies  used  can  be  as  simple  as  using  digital  pictures  and  text  in  Microsoft  PowerPoint  or  HyperStudio  or  as 
complex  as  using  digital  video  and  panoramas  in  Macromedia  Director.  The  level  of  complexity  will  depend  on  the 
technology  available  and  skills  of  the  developers. 

Virtual  field  trips  are  a hot  buzzword  for  integrating  technology  into  the  curriculum.  If  one  does  a web 
search  using  “Virtual  Field  Trip”  as  a key  phase,  hundreds  of  sites  will  be  identified.  These  sites  are  overwhelmingly 
web  based  and  provide  an  abundance  of  pictures  with  limited  information  about  locations  that  are  deemed  to  have 
educational  value.  Kmpnick,  states  that,  “The  Web  is  now  crowded  with  sites  that  are  considered  ‘Virtual  Field 
Trips’  and  they  vary  a great  deal  in  content  and  usefulness.”  She  further  states  that,  “For  teachers  who  already  have 
a curricular  unit  assembled  and  are  looking  for  online  enhancements,  these  are  adequate.  But  for  teachers  who  are 
looking  for  a source  of  new  and  exciting  curriculum  such  sites  are  only  a starting  point.  In  order  to  make  them 
useful,  an  instructor  would  need  to  develop  curriculum.”  (2000,  p.  43) 

The  approach  taken  in  this  paper  is  that  a virtual  field  trip  is  much  more  than  a web-based  presentation  of  a 
location.  Our  focus  is  to  show  how  to  design  a field  trip  based  on  curriculum  content  that  allows  the  student  to 
“learn”  from  the  field  trip  in  a way  that  is  similar  to  actually  taking  the  field  trip.  Coulter  states,  "The  key  is  for 
(instruction)  to  drive  the  technology  implementation  and  not  vice  versa,  despite  pressures  to  integrate  technology 
into  the  classroom....  virtual  field  trips  ...  enable  students  to  refine  and  extend  their  growing  understanding  as  they 
explore  other  parts  of  the  world.”  (2000,  p.  49) 

In  educational  situations  actual  field  trips  need  to  be  more  than  “just  a day  away  from  school,”  with  limited 
tangential  educational  purpose.  They  should  be  as  real  a connection  to  the  “real  world”  as  possible.  A normal  day 
field  trip  does  not  offer  sufficient  time  to  take  full  advantage  of  all  a site  has  to  offer.  Bellan  and  Sheurman  (1998) 
discuss  ideas  about  actual  and  virtual  field  trips  and  state  pitfalls  of  both.  Their  article  outlines  the  teacher 
involvement  necessary  to  alleviate  some  of  the  pitfalls.  Among  these  are  a teacher  visiting  both  the  actual  and  virtual 
sites,  the  teacher’s  plan  for  student  use  of  a virtual  site,  student  use  of  the  virtual  site  before  visiting  actual  sites,  and 
follow  up  instruction  using  both  the  virtual  and  actual  trip  experiences.  Taking  the  field  trip  on  the  computer  does 
not  eliminate  all  the  concerns  about  how  field  trips  are  used.  The  virtual  field  trip  is  usually  not  sufficient  by  itself. 
Pre-planning  the  trip  and  the  follow-up  experiences  should  be  addressed  as  part  of  the  teacher’s  curriculum.  In 
addition  we  suggest  a virtual  field  trip  should  be  developed  by  the  teacher  to  be  used  in  connection  with  an  actual 
field  trip  to  the  site. 


Definition  of  a Virtual  Field  Trip 

Virtual  field  trips  as  referred  to  in  this  paper,  are  computer-based  simulations  of  an  actual  field  trip,  which 
allows  the  user  to  vicariously  experience  the  environment  of  the  intended  location.  They  provide  the  teacher  and 
learner  the  opportunity  to  explore  aspects  of  an  actual  trip  without  leaving  the  classroom.  They  should  include  all 
elements  of  a well-designed  field  trip  and  provide  the  student  with  experiences  that  are  beyond  those  that  could  be 
obtained  from  a pamphlet  about  or  a photo  display  of  the  location.  Stainfield,  Fisher,  et.  al.  (2000)  feel  that  virtual 
refers  to  “digital  alternative  representations  of  reality”  and  that  a virtual  field  trip  is  not  an  attempt  to  create  a virtual 
reality. 


The  Power  of  Instructor  Designed  Virtual  Field  Trips 

Instructors  can  create  their  own  virtual  field  trip  with  their  students  by  using  a digital  camera.  By  planning 
what  they  would  like  to  include,  the  field  trip  can  be  designed  to  meet  the  objectives  of  the  specific  instructional 
situation.  The  site  used  for  the  virtual  trip  can  be  one  that  is  within  the  student’s  local  community.  Choosing  a site 
that  can  be  visited  by  the  class  or  a student’s  family  will  vastly  increase  the  educational  value  of  the  experience.  This 
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requires  that  teachers  must  have  control  in  creating  the  trip  to  meet  their  curricular  needs.  Student  involvement  in  the 
creation  of  the  virtual  field  trip  can  give  power  to  the  experience.  This  can  take  the  form  of;  collecting  information  at 
school,  using  aspects  of  the  technology  in  the  construction  of  the  product,  taking  the  actual  field  trip  and  gathering 
the  images  and  information  that  is  presented  in  the  virtual  field  trip.  This  type  of  involvement  focuses  student’s 
attention  and  learning  on  the  intricate  aspects  of  the  content  presented.  In  addition  it  provides  ownership  by  the 
students  of  the  project  and  focuses  their  learning  of  the  curricular  content.  Last,  but  not  least,  it  provides 
opportunities  for  students  to  use  the  technology  in  a real  world  situation,  which  addresses  the  NETS  standards  by 
preparing  them  to  function  in  a technology  rich  information  society. 

Advantages  of  Creating  a Virtual  Field  Trip 

The  main  advantage  of  a virtual  field  trip  is  that  it  can  be  used  to  meet  the  objectives  of  the  curriculum  and 
the  needs  and  ability  levels  of  the  students.  Other  advantages  include: 

• providing  opportunities  for  repeated  visitations  to  the  site  for  continued  study. 

• allowing  the  teacher  to  focus  on  one  specific  aspect  of  the  trip  at  a time. 

• providing  for  the  presentation  of  a wider  variety  of  experiences  than  may  be  possible  on  one  trip. 

• illustrating  time  sensitive  issues  that  could  not  be  viewed  on  a single  actual  trip. 

• allowing  classes  in  a different  geological  location  to  visit  and  compare  with  resources  in  their  own  area. 

• providing  integration  of  the  multiple  aspects  of  the  field  trip  into  a number  of  different  curriculum  areas. 

• allowing  for  commonality  of  experiences  by  all  participants. 

• allowing  students  to  take  a closer  look  at  areas  that  could  not  be  fully  explored  during  an  actual  field  trip. 

• using  for  assessment  purposes. 

• sharing  with  colleagues  and  parents. 

• using  repeatedly  by  the  teacher  year  after  year. 

When  teachers  use  a virtual  field  trip  in  conjunction  with  an  actual  field  trip  to  the  same  site,  a virtual  trip 
can  serve  as  a motivator  for  the  trip,  or  as  an  advanced  organizer  for  the  day’s  activities.  It  also  encourages  learners 
to  plan  and  prepare  for  activities  to  be  carried  out  on  the  trip  or  can  provide  a simulation  for  students  who  may  not 
have  been  able  to  attend  the  actual  field  trip. 

Limitations  of  a Virtual  Field  Trip 

There  are  two  major  limitations  to  using  a virtual  field  trip.  The  first  is  the  availability  of  a field  trip  that 
meets  the  objectives  of  the  curriculum.  Commercially  available  and  Web  based  field  trips  are  designed  for  a large 
audience  and  may  need  to  be  adapted  to  meet  an  individual  instructors  curriculum  needs.  The  second  limitation 
focuses  on  the  designer’s  expertise  in  the  content  area  and  appropriate  pedagogy  for  the  field  trip. 

Barriers,  and  the  Paths  over  Them 

The  main  limitation  in  creating  a virtual  field  trip  is  the  time  needed  to  create  the  experience.  The  teacher 
who  wants  to  create  a virtual  field  trip  needs  to  find  ways  of  involving  others  such  as;  colleagues,  community 
members,  students  and  parents.  If  the  teacher  views  the  design  of  the  field  trip  as  dynamic,  constantly  evolving, 
construction  of  both  the  actual  and  virtual  field  trips  will  grow  over  time. 

Because  curriculum  design  requires  content  expertise,  another  limitation  is  the  creator’s  knowledge  of  the 
content  area  of  the  field  trip  and  the  curricular  objectives.  As  teachers  become  involved  with  the  design  they  often 
need  to  extend  their  content  knowledge,  thus  increasing  the  time  necessary  to  design  the  experience.  On  the  positive 
side,  they  can  get  so  involved  that  their  personal  knowledge  increases  and  it  becomes  a professional  development 
experience. 

A third  limitation  is  the  availability  of  the  technology.  However  with  the  increasing  availability  of 
technology  in  the  schools,  homes  and  work  places  this  is  becoming  less  of  a barrier.  Coupled  with  the  availability  of 
the  technology  is  the  teacher’s  ability  to  use  the  technology  effectively.  Again  this  can  become  a learning  situation 
and  thus  increases  the  time  to  create  the  product.  This  limitation  can  be  minimized  by  the  involvement  of  colleagues, 
community  members,  students  and  parents. 

When  evaluating  the  effectiveness  of  a virtual  field  trip  one  should  consider  both  the  appropriateness  and 
the  effectiveness  of  the  technology  used  in  the  presentation  of  the  content.  Children  frequently  experience  learning 
situations  indirectly  thought  technology  by  the  use  of  pictures,  time-lapse  photography  and  interactive  non-linear 
experiences  that  allow  the  learner  to  follow  their  interest  and  to  revisit  locations  in  the  program  as  their  interest 
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grows.  Technology  should  not  be  seen  as  a replacement  for  experiences  but  as  an  enhancement  of  these 
experiences.  Technology  can  be  used  to  enhance  the  senses,  build  interest  and  excitement  or  to  review  and  analyze 
experiences.  Digital  cameras  provide  the  technology  necessary  to;  record  experiences  for  later  analysis  without 
damaging  the  environment,  document  cause  and  effect  relationships,  record  changes  over  time,  document  unique 
events,  record  images  that  give  historical  perspectives,  open  vistas  of  limited  access  to  all  learners,  and  reviewing 
and  analyzing  experiences. 


Conclusion 

A virtual  field  trip  that  resides  on  a storage  medium  and  not  on  the  WWW  has  advantages  in  its 
effectiveness.  The  program  is  not  dependent  on  access  to  or  the  vagaries  of  the  Web,  it  runs  faster,  it  is  always 
accessible,  and  it  focuses  the  student’s  attention  on  the  task  at  hand. 

This  paper  uses  the  terms  teacher  created,  creator,  and  assembler  in  an  apparently  interchangeable  manner. 
This  is  due  to  the  principle  that  while  the  teacher  is  the  driving  force  in  the  design  and  creation  of  the  virtual  field 
trip;  the  tasks  can  be  done  in  conjunction  with  students,  parents,  interested  community  members  and  colleagues.  The 
technologies  used  to  develop  and  create  virtual  field  trips  are  commonly  found  in  public  schools  today.  This  process 
of  creating  virtual  field  trips  can  be  as  simple  as  using  digital  pictures  and  text  in  Microsoft  Power  Point  or 
HyperStudio  or  as  complex  as  using  digital  video  and  panoramas  in  Macro  Media  Director.  The  technology  skills 
necessary  in  the  creation  of  the  field  trip  all  are  within  the  scope  of  the  National  Educational  Technology  Standards 
(NETS)  for  Teachers,  and  many  are  within  the  scope  of  the  NETS  for  students. 
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Abstract:  This  study  looks  at  team  collaboration  in  marketing  a University  blended-delivery 
program  successfully.  This  presentation  reports  action  research  which  investigated  the 
development  and  implementation  of  a marketing  program  that  successfully  boosted  enrollments 
over  49%  in  a blended  University  blended  university  program.  From  its  inception  through  the 
second  year  of  operation,  program  enrollment  increased  due  to  the  marketing  program  based  on 
an  effective  approach  grounded  in  simple  marketing  principles.  Lessons  learned  along  the  way 
can  shed  new  light  on  how  to  improve  future  marketing  of  new  programs  and  successful 
adoption. 

KEYWORDS:  Action  Research;  Blended  Courses;  Marketing  Strategies;  Distance  Learning 

Introduction 

This  study  looks  at  team  collaboration  in  successfully  marketing  a University  blended- 
delivery  program.  The  researchers  discuss  lessons  learned  in  marketing  education  programs  from 
a large  public  university  in  the  southwestern  United  States.  The  study  focuses  on  marketing, 
action  research  and  collaboration  as  a means  to  accomplish  innovation  diffusion,  and  as  a model 
for  change  integration  of  new  university  programs  (Bourner,  Katz  & Watson,  2000;  Isgar,  1995, 
Larson  & LaFasto,  1989). 

The  research  team  observed  a marketing  plan  implemented  by  their  university  to  gain  a 
larger  market  share  in  a fertile  potential  academic  market.  From  the  inception  of  the  plan  through 
the  second  year  of  operation,  program  enrollment  doubled  due  to  the  implementation  of  the 
marketing  program  which  was  simple  and  direct.  Throughout  the  process  the  project  was 
monitored  through  an  instructional  design  and  action-research  perspective,  to  facilitate  field- 
based  research  for  future  marketing  university  initiatives.  The  results  were  successful,  largely, 
the  researchers  found,  because  the  university  delivered  what  it  promised. 

The  study  utilizes  change  theory  to  assess  how  successful  innovation  can  quietly 
assimilate  into  the  culture  of  the  academic  environment  without  mandates  from  management 
(Fullan,  1991,  1994;  Gannon-Cook,  Crawford  & Varagoor,  1999;  Robinson,  1995).  Once  the 
innovation  is  adopted,  it  becomes  less  threatening.  Templates,  training,  and  instructional  tools 
can  help  the  electronic  innovations  become  adopted,  ultimately  turning  the  adopters  into 
assimilators.  Once  the  assimilating  or  “morphing”  of  the  innovation  takes  root,  it  may  still  take 
some  time  for  the  “old  guard”  resistors  to  adopt,  but  the  wave  of  change  can  carry  latecomers 
along  with  the  critical  mass  of  adopters.  As  it  increases  usage,  a “tipping  point”  (Gladwell,  2002) 
occurs,  and  the  innovation  becomes  part  of  the  fiber  of  the  evolving  culture  of  the  community. 
Rapid  prototyping  can  then  be  used  to  continue  the  marketing  program  and  the  collaborative 
efforts  can  be  promulgated  and  nurtured,  both  electronically  and  face-to-face. 
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The  importance  of  the  action  research,  of  painstaking  observation,  and  noting  each  team 
member’s  style  of  implementation,  provides  crucial  data  throughout  each  step  of  the  study.  The 
results  of  this  study  are  supported  by  the  university  data  reported  one  year  after  the  study:  49% 
increases  in  the  blended  university  degree  programs  introduced  into  the  targeted  market 
geographic  areas  as  a direct  result  of  the  marketing  campaign.  Researching  and  journaling  the 
effects  of  team  collaboration  in  the  marketing  of  programs  proved  beneficial  to  both  the 
university  administration  and  to  the  targeted  communities  in  need  of  new  educational  programs. 

One  finding  in  this  study  was  that  “word  of  mouth”  can  turn  out  to  be  a critical  ingredient 
to  the  success  of  a marketing  program.  In  addition,  combined  delivery  methods  and  support  by 
the  sponsoring  institution  are  crucial  to  the  embedding  of  the  innovation  into  the  target 
(educational)  market.  What  is  beneficial  for  the  sponsoring  institution  is  the  body  of  research 
from  the  target  program  that  can  provide  important  data  to  lead  to  the  development  of  both 
marketing  and  online  and  blended  learning  templates  for  future  marketing  and  educational 
product  campaigns.  The  results  of  this  study  could  have  residual  effects  on  both  the  sponsoring 
institution  and  the  targeted  educational  community  that  affect  the  cultures  of  both  the  institution 
and  organization,  as  well  as  contribute  to  the  body  of  literature  on  the  marketing  of  educational 
products  and  services. 


Literature  Review 

Marshall  McLuhan  (1964)  predicted  a “global  village”,  over  fifty  years  ago,  and  today, 
we  are  living  in  it.  In  this  global  community,  it  is  crucial  to  develop  personal  and  professional 
electronic  development  frameworks.  Recent  research  has  posited  that  there  may  be  a relationship 
between  an  organization’s  level  of  capability  in  delivery  of  training  and  the  barriers  set  up  to 
detain  it.  One  of  the  biggest  obstacles  is  the  entrenched  culture  of  the  organization  itself  (Berge 
& Muilenburg,  2001,  Shaik,  2005).  Often  the  challenges  to  the  implementation  of  innovation, 
such  as  electronic  instruction,  come  from  the  very  establishment  committed  to  its  adoption. 
Organizations  can  be  at  different  stages  of  maturity  regarding  their  capabilities  to  conduct 
electronic  education,  but  too  often  the  organization  is  hurdling  forward  at  such  a brisk  pace  that 
training  is  developed  and  delivered  without  an  audit  trail  of  testing  (2001).  In  many  instances, 
little  formative  test-marketing  is  conducted  when  new  programs  are  initiated,  so  action  research 
can  offer  a “failsafe”  opportunity  for  test-marketing  that  can  also  provide  valuable  insights  into 
the  needs  of  a community  without  the  university  administration  losing  too  much  valuable  effort 
or  money. 

Many  academic  administrators  seem  to  think  that,  when  it  comes  to  creating  and 
marketing  distance  education  programs,  “if  you  build  it,  they  will  come”  (Robinson,  P.  1984). 
However,  in  the  light  of  recent  investigations  into  ineffective  corporate  university  online 
programs,  it  may  be  more  important  than  ever  for  administrators  to  evaluate  the  reasons  why 
they  want  to  increase  their  online  programs  and  learn  the  lessons  taught  by  successful  marketers. 
Research  on  marketing  new  products  conducted  at  Harvard  has  revealed  interesting  data  on  what 
kinds  of  marketing  influences  people  to  purchase  products  and  these  findings  could  shed  new 
light  on  the  importance  of  “the  people  factor”  on  the  successful  marketing  of  new  products 
(Zaltman,  1997).  The  Zaltman  study  pointed  to  “word  of  mouth”,  most  notably  in  the  form  of 
testimonials,  were  more  successful  in  marketing  a new  product  than  media  advertisements.  The 
good  news  is  that  people  will  talk  about  something  they  are  excited  and  feel  good  about,  but  the 
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bad  news  is  that  they  will  also  talk  about  bad  experiences  too.  So  it  is  important  to  make  sure  that 
the  product  can  be  delivered  consistently  in  order  to  get  and  keep  positive  testimonials  from 
satisfied  customers,  and  these  findings  can  also  apply  to  the  field  of  education  as  well  as 
business,  as  posited  by  results  of  this  study. 

Adoption  of  an  innovation  seems  to  occur  more  effectively  when  the  innovation  becomes 
assimilated  into  the  culture  of  the  environment  (Fullan,  1991;  Fullan,  1994;  Robinson,  1995; 
Rogers,  1995;  National  Council  for  Educational  Technology,  1995).  Once  the  innovation 
becomes  less  threatening,  employees,  in  this  case  teachers,  become  more  comfortable  and  less 
threatened.  As  more  employees  (teachers)  begin  to  use  the  innovation,  they  adapt  their 
workstyles  to  use  it  and  also  begin  encouraging  their  peers  to  try  it.  Adoption  then  comes  next, 
so  it  is  important  to  reinforce  participants  and  encourage  their  progress  throughout  this  step  and 
the  entire  process  in  order  to  integrate  the  innovation  (Robinson,  1995).  But  most  important  is 
the  need  to  deliver  what  was  promised  to  the  target  market  and  to  be  sure  the  key  supporters  and 
early  adopters  know  that  they  are  receiving  what  was  promised.  This  is  perhaps  the  most  crucial 
lesson  learned  throughout  the  study:  it  is  not  as  important  to  have  a new  and  innovative 
“product”  (and  it  is  somewhat  important  to  have  some  new  “product”)  than  it  is  to  deliver  what 
was  promised.  In  other  words,  the  potential  supporters  do  not  want  to  be  promised  “blue  sky” 
then  delivered  something  that  bears  no  resemblance  to  what  was  promised,  or  worse  yet,  to  not 
have  the  “product”  delivered  at  all.  (In  this  instance  the  “product”  was  an  entire  Master’s 
Program  that  could  be  delivered  in  both  face-to-face  and  blended  environments  to  a school 
district  50  miles  away  from  the  university  “pitching”  the  new  program). 

One  study  on  change  related  to  new  technologies,  standards  and  assessments  (Bourner, 
Katz  & Watson,  2000),  looked  at  what  new  directions  universities  could  take  with  respect  to 
technology  and  education  in  a global  community  and  eLearning  development  frameworks.  The 
research  did  not  analyze  technology,  best  practice,  or  which  factors  could  influence  participation, 
but  focused  on  internal  development  frameworks,  particularly  with  respect  to  collegiality, 
professional  training  of  faculty,  personal  collaborations  and  professional  collaborations 
(Boumer,  Katz  & Watson,  2000).  The  study  seemed  to  reinforce  what  the  researchers’  of  this 
study  found,  and  that  is,  there  needs  to  be  consistency  in  promising  specific  deliverables  and 
being  sure  those  deliverables  are  consistently  provided.  In  the  instance  of  this  study,  the 
marketing  representatives  worked  well  with  school  district  administration  and  prospective 
students,  and  did  so  consistently,  so  a bond  of  trust  was  fonned  that  proved  invaluable  to  the 
support  and  delivery  of  the  innovation. 

A interesting  piece  of  literature  from  New  Yorker  staff  writer  Malcolm  Gladwell  (2002) 
supports  the  collaboration  concept,  or  at  the  least,  a contagion  of  communication,  and  points  to 
the  “word  of  mouth”  factor  as  being  one  of  the  most  important  to  the  ongoing  success  of  an 
innovation.  Gladwell  purports  that  there  are  three  identifying  characteristics  to  successful 
products  or  marketing  efforts.  These  are: 

one,  contagiousness;  two,  the  fact  that  little  causes  can  have  big 
effects;  and  three,  that  change  happens  not  gradually  but  at  one 
dramatic  moment-(these)  are  the  same  three  principles  that  define 
how  measles  moves  through  a grade-school  classroom. . . 
of  the  three  (identifying  characteristics),  the  third  trait- 
the  idea  that  epidemics  can  rise  or  fall  in  one  dramatic  moment- 
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is  the  most  important,  because  it  is  the  principle  that  makes  sense 
of  the  first  two  and  permits  the  greatest  insight  into  why  modern 
change  happens  the  way  it  does.  The  name  given  to  that  one 
dramatic  moment  in  an  epidemic  when  everything 
everything  can  change  all  at  once  is  (called)  the  “Tipping  Point.”  (p.9) 

But  the  tipping  point  can  be  a pendulum;  swinging  both  for  and  against,  the  innovation,  so  it  is 
important  to  assure  consistency  in  the  ongoing  delivery  of  a new  product  over  a longer  period 
than  an  initial  trial  “honeymoon”  period  of  time.  Short-sighted  timelines  for  implementation  of 
new  programs  can  have  exactly  the  opposite  of  the  desired  effect,  but  once  the  innovation  gets  a 
“bad”  or  “negative”  reputation,  it  is  much  more  difficult,  and  sometimes  impossible,  to  regain 
lost  territory.  A number  of  other  studies  also  support  the  importance  of  collaboration  and  the 
keeping  of  promises  (Clark,  Moss,  Goering,  Herter,  & Wascha,  1996;  Faber  & Green,  2001; 
Giarratano  & Gannon-Cook,  1999;  Havernik,  Messerschmitt,  & Vandrick,  1997;  Haverschmidt, 
Smithl998;  John-Steiner,  Weber,  & Minnis,  1998;  Larson  & LaFasto,  1989;  Robinson,  1995; 
Shaik,  2005;  Stribiak  & Paul,  1998). 

Another  study  that  addressed  adoption  of  an  innovation  reviewed  obstacles  to  distance 
education  (eLearning)  implementation  in  higher  education.  This  study  posed  that  business  and 
academic  organizations  can  be  at  different  stages  of  maturity  regarding  their  capabilities  to 
integrate  an  innovation,  in  this  case,  eLearning  (Berge  & Muilenburg,  2001).  The  survey  posited 
there  may  be  a relationship  between  an  organization’s  (or  community’s)  level  of  capability  in 
adoption  and  the  barriers  to  adoption.  The  greatest  obstacles  in  the  study  were  reported  to  be: 

(1)  Employee  (faculty)  compensation  and  time  taken  to  do  innovation; 

(2)  organizational  change  of  the  culture  of  an  organization.  In  the  early  stages  of  integration,  the 
struggle  mainly  focused  upon  changing  the  culture  of  the  institution; 

(3)  the  lack  of  technical  expertise  and  support; 

(4)  evaluation/effectiveness  issues  increase  as  innovation,  in  this  case  educational  programs,  is 
increasingly  used;  and, 

(5)  user  support  services  necessary  for  innovation. 

The  researchers  in  the  southwestern  university  marketing  study  found  these  five  items  to  be 
challenges  in  the  implementation  of  their  marketing  program  because  their  target  group  was  a 
school  district.  Teachers  are  usually  time-challenged,  entrenched  in  a culture  resistant  to  change, 
without  tools  and  support  to  enforce  and  integrate  the  innovation  (Berge  & Muilenburg,  2001; 
Bonk,  2001;  Bonk,  Kirkley,  Hara,  & Dennen,  2001;  Clark,  Moss,  Goering,  Herter,  & Wascha, 
1996).  This  knowledge  helped  the  researchers  to  amend  their  marketing  plan  to  include  more 
face-to-face  support  to  nurture  new  candidates  to  join  the  Master’s  and  certificate  programs,  and 

to  continue  maintaining  that  support  by  initiating  reasons  to  visit  sites,  follow-up  with  requests, 
and  provide  technical  expertise  to  program  participants. 

Research  on  team  management  and  of  other  types  of  innovations,  including  eLearning 
projects,  pointed  to  the  importance  of  the  innovation’s  adaptation  and  reinforcement  in  both 
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environmental  and  cultural  contexts  (Kaufman,  2000;  Larson  & LaFasto,  1989  ).  This  can  be 
accomplished  by  reinforcement  of  collaborations,  by  delivering  the  product  promised,  and  by 
providing  service  and  encouragement  to  the  customers  (in  this  case  the  teachers  and  school 
district  administrators  who  use  and  support  the  innovation)  (Faber  & Green,  2001).  In  this 
instance,  this  meant  the  southwestern  university  needed  to  reinforce  its  commitment  to  its 
collaborators  that  it  would  deliver  and  maintain  these  blended  graduate  and  certificate  courses.  If 
these  things  are  done,  the  delivery  of  the  product  promised,  the  provision  of  consistent  and 
reliable  service,  and  the  encouragement  and  support  to  the  adopters,  then  the  innovation  becomes 
further  integrated  into  the  fiber  of  both  the  provider’s  (the  southwestern  university)  and  the 
customer’s  (the  school  district)  culture  (Kaufman,  2000;  Robinson,  1995;  Rogers,  1995;  Rogers 
& Shoemaker,  1971;  Stribiak  & Paul,  1998;  Varagoor,  1998;  Wolcott,  2002). 

Adoptive  Behaviors,  self-determinism,  competence,  tendency  for  success  and  tendency  to 
avoid  failure,  all  contribute  to  successful  participation  in  an  innovation,  in  this  case,  the 
marketing  of  the  Master’s  Degree  and  certificate  programs.  The  five  attributes  of  innovation, 
developed  by  Rogers  and  Shoemaker  (1971),  are: 

• relative  advantage; 

• compatibility; 

• complexity; 

• trialability; 

• observability  and  assess  for  the  potential  adoption  capability  of  the  innovation  within  the 

culture  of  the  population  sampled. 

These  attributes  also  serve  well  for  action  research.  Findings  of  this  study  suggest  that  one 
primary  requisite  to  adoption  of  an  innovation  is  that  it  must  be  perceived  as  simple  to  use  and 
maintain  if  it  is  to  be  successfully  implemented  (Allan  & Wolf,  1978;  Gannon  Cook  & 

Crawford,  2001;  Robinson,  1995).  Another  requisite  is  that  the  innovation  must  be  supported  and 
maintained  by  the  sponsoring  provider.  Again,  the  delivery  of  the  product  promised  and  the 
ongoing  accommodation  of  that  product  are  paramount  to  the  innovation’s  ongoing  success. 

Methodology 

This  study  utilized  a qualitative  research  methodology,  action  research,  to  apply  the 
observations  used  in  educational  research  to  a higher  education  marketing  program.  One  of  the 
researchers  acted  as  action  researcher  as  well  as  marketer  and  was  embedded  in  the  marketing 
team.  The  marketing  team  consisted  of  the  department  chairperson,  one  marketing  specialist, 
several  volunteer  professors  who  actively  marketed  the  program,  and  a remote-site  campus 
director  who  agreed  to  offer  the  Master’s  and  certificate  programs  in  blended  online  and  face-to- 
face  classes. 

Action  Research 

Action  research  is  “the  practical  application  of  the  scientific  method  or  other  forms  of 
disciplined  inquiry  to  the  process  of  dealing  with  everyday  problems”  (Vockell  & Asher,  1995, 
p.  445).  The  tenn  action  research  evolved  from  actual  research  conducted  informally  in 
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Kindergarten  through  twelfth  grade  (K-12)  classrooms  by  teachers  who  needed  to  find 
expeditious  solutions  to  classroom  problems  (Gay,  1987,  p.  8).  Data  collection  was  observed  by 
the  teacher  as  researcher  who  would  journal  the  process  of  discovery  and  provide  a chronicle  that 
would  describe  the  steps  taken  as  well  as  the  solution  that  solved  the  problem. 

In  earlier  action  research  studies,  the  teacher  often  began  the  study  with  a new  classroom 
initiative  that  needed  documentation,  then  the  teacher  organized  and  analyzed  the  qualitative  data 
from  the  study  throughout  the  initiative’s  implementation  and  reported  the  findings  upon 
completion  of  the  initiative’s  implementation.  Through  the  last  fifteen  years,  action  research  has 
come  to  be  a reliable  source  of  data  and  has  provided  valuable  insights  to  teachers  in  the  field 
and  has  been  acknowledged  as  an  excellent  source  for  important  archival  data.  It  is  now  used  not 
only  in  both  K-12  settings,  but  in  both  higher  educational  and  training  and  human  perfonnance 
settings.  “In  action  research  the  emphasis  is  more  on  what  practitioners  do  than  what  they  say 
they  do”  (Avison,  Lau,  Meyers,  & Nielsen,  1999,  p.  93).  The  action  research  approach  is  “value- 
driven,  attuned  to  power  issues,  committed  to  stakeholder  participation,  and  action-oriented” 
(Nelson,  2004,  p.389). 

The  origin  of  action  research  has  been  attributed  to  organizational  trainers  such  as  Kurt 
Lewin  (1951)  as  a collection  of  problem-solving  cycles  for  improving  organizations.  “The  tenn 
‘action’  captured  the  notion  of  a disciplined  inquiry  in  the  context  of  focusing  efforts  to  improve 
the  quality  of  an  organization  and  its  performance”  (National  Literacy  Secretariat,  n.  d.).  The 
research  process  involved  identification  of  the  project;  analysis;  planning;  observing;  evaluating; 
amending;  and  beginning  the  cycle  again. 

The  integration  of  theory  and  practice  is  the  key  to  action  research.  (National  Literacy 
Secretariat,  n.  d.). 

The  value  of  action  research  is  that  it  represents  a scientific 
approach  to  problem  solving  that  is  considerably  better  than 
change  based  on  the  alleged  effectiveness  of  untried  procedures, 
and  infinitely  better  than  no  change  at  all.  It  is  a means  by  which 
concerned  school  personnel  can  attempt  to  improve  the  educational 
process,  at  least  within  their  environment. . .True  progress 
requires  the  development  of  sound  theories  having  implications 
for  many. . .not  just  one  or  two.  One  sound  theory  that  includes  10 
principles  of  learning  may  eliminate  the  need  for  hundreds  of 
would-be  action  research  studies.  Given  the  current  status  of 
educational  theory,  however,  action  research  provides  immediate 
answers  to  problems  that  cannot  wait  for  theoretical  solutions. 

(Gay,  1987,  p.  9) 

The  Marketing  Study 

In  this  study,  the  initial  marketing  program  was  designed  to  appeal  to  a limited 
geographic  radius  so  that  the  marketing  program  effectiveness  could  be  tracked  to  a specific 
marketing  code  and  also  to  a physical  campus  location.  The  program  campaign  targeted  a very 
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large  school  district  within  fifty  miles  of  the  university  and  the  initial  campaign  was  to  market  a 
Masters  degree  in  Instructional  Design  and  a certificate  program  to  the  school  district. 

The  marketing  campaign  took  place  over  a one  year-period,  and  the  implementation  of 
the  marketing  campaign  was  rolled  out  over  the  second  six  months  and  following  year.  The 
faculty  from  the  southwestern  university  who  elected  to  participate  in  the  marketing  team  were 
given  six  months  initially  to  design  the  basic  marketing  campaign,  test  and  implement  it.  The 
team  was  provided  with  a very  small  marketing  budget,  enough  to  make  copies  of  flyers  and  to 
sponsor  some  in-person  presentations  to  the  school  district  that  included  cookies  and  soft  drinks. 
They  were  also  provided  with  some  travel  reimbursements  for  mileage  to  the  school  district  to 
which  the  marketing  campaign  would  be  targeted.  Rapid  marketing  campaign  prototypes  were 
developed  and  several  onsite  promotional  presentations  were  held  to  encourage  school  district 
participation  in  the  marketing  program.  The  action  researcher  provided  the  reports  and  journals 
to  the  team  at  the  end  of  the  initial  trial  period,  then  the  university  monitored  and  reported  the 
results  of  the  program  at  the  end  of  the  first  and  second  year  (of  the  program). 

Analysis  of  the  Action  Research  Conducted  in  this  Study 

Since  the  plan  took  longer  than  originally  scheduled,  adjustments  in  the  rollout  cycle  of 
the  program  implementation  were  made  to  accommodate  the  extended  timeframe  program 
execution. 

The  first  discovery  by  the  lead  marketing  faculty  team  member  was  also  the  first  project 
setback;  it  was  the  uncovering  of  a problem  that  could  have  been  a “deal  breaker”,  a prior 
university  marketing  program  that  had  devastating  effects  on  the  same  community  targeted  for 
this  study.  That  first  marketing  program  had  begun  very  successfully  and  had  continued  for  two 
years,  but  was  then  adjudged  by  the  university  to  be  too  expensive  to  maintain,  so  it  was 
summarily  dropped  with  little  notice  to  the  community.  So  when  that  program  was  pulled,  the 
students  were  forced  to  take  online  courses  from  their  university  when  they  were  available,  or 
travel  50  miles  to  the  university  to  complete  their  degrees.  This  had  caused  the  school  district 
both  emotional  and  financial  pain  for  the  teachers  in  the  program.  Many  ended  up  being  forced 
to  discontinue  the  program.  This  also  caused  considerable  chagrin  and  the  embarrassment  to  the 
school  district:  the  administrators  felt  they  had  been  “baited  and  switched.” 

Ironically,  despite  the  huge  potential  for  problems  presented  by  this  prior  disastrous 
experience,  the  school  district  was  willing  to  enter  an  agreement  with  the  southwestern  university 
once  again  to  help  their  students  attain  the  Masters  degree  and  certificates.  The  district 
administrators  expressed  hope  that  the  new  university  staff  involved  in  the  marketing  campaign 
would  deliver  what  the  university  promised  this  time.  The  researchers  documented  every 
meeting  and  the  results,  then  provided  a chronicle  of  the  rapid  prototypes  of  the  program. 

The  action  researcher  reported  the  following  observations  and  findings: 

• The  marketing  plan  was  projected  to  be  offered  to  an  educational  market  in  need  of  the 

Master’s  Degree  in  Instructional  Technology  and  Instructional  Design  (ID)  Certificate 

Program; 
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• The  data  on  the  educational  market  covered  basic  infonnation  and  the  marketing  plan 
was  created  in  response  to  requests  from  individuals  in  that  area,  so  there  was  reasonable 
belief  that  the  implementation  of  this  marketing  plan  could  yield  enough  students  to 
people  the  Master’s  Degree  and  very  possibly  the  ID  Certificate  Program; 

• Obstacles  were  greater  than  anticipated,  but  with  rapid  prototyping  of  the  marketing  plan, 
objections  were  addressed  and  the  plan  was  implemented  successfully; 

• The  recorded  dialogs  depicted  an  evolving  fiduciary  relationship  between  the  university 
and  school  district  that  forged  a stronger  bond  than  in  years  past. 

The  final  reports  of  the  university  at  the  end  of  the  first  and  second  year  of 
implementation  showed  a 49%  increase  in  the  enrollments:  an  increase  of  24%  over  the  last  five 
months  of  the  first  year  (January  through  May  2003);  and  an  increase  of  25%  over  the  second 
year  (May  2003  through  May  2004). 

What  Succeeded  in  the  Study 

The  initial  project  took  six  months  longer  than  originally  anticipated,  largely  due  to  the 
hesitancy  by  the  school  district  to  initially  participate  in  the  marketing  program  based  on  their 
prior  experience  with  the  southwestern  university.  However,  after  trust  was  cultivated,  the  initial 
rollout  of  both  the  Master’s  and  certificate  programs  were  completed  by  the  end  of  the  first  year 
and  revisions  were  done  on  an  ongoing  basis  by  the  marketing  team.  The  researchers  attributed 
the  success  of  the  Master’s  and  certificate  programs  to  the  university’s  delivery  of  exactly  what 
was  promised  to  the  school  district  administrators. 

Discussion 

Lifelong  learning  is  going  to  become  more  prominent  for  most  21st  century  citizens  as 
more  people  compete  for  fewer  jobs  in  a global  job  marketplace  (United  States  Distance 
Learning  Association,  2001).  So  incentives  that  offer  educational  opportunities,  including  degree 
or  advanced-degree  seeking  opportunities,  and  skill-specific  training  certificates,  will  continue  to 
increase  with  this  need.  It  will  be  important  to  find  innovative  and  cost-effective  ways  to  market 
new  programs  and  reach  as  many  candidates  as  possible  for  these  programs.  While  research  on 
marketing  should  include  “how  to”  avoid  failure,  its  primary  purpose  should  be  to  study  how  to 
anchor  positive  and  sustained  outcomes  for  new  educational  innovations  (Robinson,  1995;  Schott 
& GannonCook,  2001). 

The  Institute  for  Higher  Educational  Policy  (2000)  established  several  bookmarks  for 
successful  technology  diffusion,  and  these  bookmarks  can  also  apply  to  most  innovations: 

• technical  assistance  throughout  the  process  of  implementation; 

• (personal  and  ongoing)  assistance  in  transition  to  DE  (the  innovation); 

• continued  assistance  and  DE  (innovation)  training  (Bower,  2000;  Institute  for  Higher 

Educational  Policy,  2000). 
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Rogers  (1995)  found  that  certain  attributes  predict  adoption  of  technology:  relative  advantage, 
compatibility,  complexity,  trialability,  and  observability  (Wolcott,  1997).  When  the  innovation  is 
perceived  to  be  better,  and  consistent  with  the  adopter’s  needs  and  values,  the  faculty  member 
will  be  more  likely  to  try  the  innovation.  Once  tried,  if  the  result  is  successful,  then  it  will  be 
adopted.  These  principles  certainly  held  true  for  this  study;  the  action  research  collected  from  the 
marketing  team  supported  the  findings  of  the  research  cited  herein,  that  positive  expectancy  led 
to  adoption,  delivery  of  the  promised  product  was  met  with  expectancy,  and  consistent  delivery 
of  the  product  led  to  outcomes  of  user-satisfaction  and  adoption  (Berge  & Muilenburg,  2001; 
Bourner,  Katz  & Watson,  2000;  Jacob  & Hellstrom,  2001;  John-Steiner,  Weber,  & Minnis, 

1998). 


Summary 

The  first  step  towards  successful  acceptance  and  integration  of  an  innovation  introduced 
through  a marketing  campaign  is  to  obtain  feedback  and  deliver  the  product  that  was  promised. 
While  this  is  the  most  obvious  of  conclusions,  it  seems  to  be  one  that  often  escapes  implementers 
of  innovations.  And  failure  to  consistently  deliver  the  product  promised  may  result  in  loss  of 
confidence  and  even  prevent  future  opportunities.  So,  while  this  is  a simple  lesson  in  marketing, 
the  moral  of  the  story  is  exactly  that. . .there  is  a moral:  consistently  deliver  what  is  promised,  and 
if  possible,  perform  even  better  than  expected.  The  researcher  and  participants  in  this  study 
would  also  recommend  including  action  research  in  any  marketing  studies.  Action  research,  if 
included  in  the  initial  marketing  campaign,  can  expand  upon  the  body  of  knowledge  in  the 
research  areas  of  innovation,  and  diffusion  of  technology.  The  learning  environments  in  the  next 
millennium  could  greatly  benefit  from  these  studies. 
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Abstract 

This  paper  delineates  our  current  practices  by  sharing  experiences  of  using  synchronous  communication 
tools  for  instructional  purposes.  We  argue  that  both  synchronous  and  asynchronous  interactions  in  distance 
education  are  able  to  contribute  a synergetic  effect  to  student  learning  achievement.  While  this  technological 
symbiosis  is  correlated  with  administrative  support  and  resources,  instructors  are  advised  to  look  into  freeware 
programs  on  the  market  and  to  meditate  on  the  instructional  use  of  these  real-time  communication  tools. 
Recommendations  for  practitioners  are  outlined. 

Introduction  & Background 

With  the  level  of  synchronicity  now  available  to  virtually  anyone  with  access  to  a telephone  and  a 
computer,  one  would  expect  that  the  technology  that  revolutionized  the  way  businesses  communicate,  would  also 
find  its  way  into  distance  education  programs.  Nevertheless,  asynchronous  communication  continues  to  dominate 
the  distance  education  scene.  From  interesting  experiment  to  exciting  frontier,  desktop  videoconferencing  is  quietly 
moving  from  revolution  to  evolution,  and  it  is  time  we  gave  it  a second  look. 

When  desktop  videoconferencing  made  its  debut  in  the  mid  1990s,  it  appeared  that  Web-based  distance 
education  was  on  the  verge  of  a major  communications  revolution.  However,  after  a brief  period  of  experimentation, 
desktop  videoconferencing  was  abandoned  by  most  distance  education  providers  before  it  had  a chance  to  fully 
evolve.  Initial  tests  were  plagued  by  setbacks  including  expensive  hardware  and  software,  poor  frame  rates  and 
audio  reception,  jerky  or  frozen  images,  limited  bandwidth,  limited  capabilities,  and  incompatibility  issues.  As 
asynchronous  Web-based  distance  education  programs  flourished,  synchronous  programs  faded.  However,  within 
the  past  five  years,  several  key  technological  issues  have  been  resolved,  making  synchronous  communication  less 
expensive,  more  powerful,  and  available  to  more  people.  This  renewed  viability  suggests  that  distance  educators 
give  synchronous  communication  tools  a second  look  to  determine  how  they  can  contribute  to  distance  education, 
establish  how  they  should  be  integrated  into  a coherent  online  pedagogy,  and  justify  the  rationale  for  supporting 
them  in  distance  education  courses. 


Review  of  Literature 


From  Revolution  to  Evolution 

A revolution  is  a drastic  and  far-reaching  change  in  ways  of  thinking  and  behaving.  If  the  change  is  slow, 
gradual,  or  progressive  it  is  called  evolution.  Despite  their  slow  start,  IP-based  audio  and  video  conferencing  tools 
have  the  potential  of  revolutionizing  the  way  distance  education  courses  are  delivered.  However,  due  to  convenience 
as  well  as  other  factors,  asynchronous  communication  has  been  institutionalized  into  the  vast  majority  of  Web-based 
distance  education  programs.  Yet,  according  to  Haefner  (2000),  synchronous  and  asynchronous  communication  can 
work  symbiotically  in  online  classes.  In  an  opinion  paper  titled  The  Importance  of  Being  Synchronous,  Haefner 
wrote: 

It  is  hard  to  imagine  teaching  without  both  these  modes  of  interacting  with  our  students,  and  I can't 
believe  many  teachers  would  want  to  handicap  their  teaching  by  relying  on  just  one  of  them.  But 
anecdotal  evidence  suggests  many  online  teachers  do  exactly  this,  and  rely  almost  exclusively  on 
asynchronous  communication. . ..Computer-aided  instruction  that  is  solely  asynchronous  cannot 
possibly  convey  any  kind  of  immediacy.  If  it  takes  days,  or  even  hours,  for  students  to  get  a response  to 
a question,  many  students  will  lose  the  intellectual  thread  - and  the  urge  to  follow  it.  5) 
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The  Right  Tool  at  the  Right  Time 


As  with  most  instructional  tools,  there  are  many  advantages,  and  potentially  disadvantages,  to  synchronous 
and  asynchronous  communication.  The  key  to  successful  implementation  is  to  know  when  to  use  the  right  tool  at  the 
right  time  for  the  right  purpose.  Collis  (1996)  identified  four  significant  advantages  of  synchronous  over 
asynchronous  systems: 

• Motivation  - synchronous  systems  focus  the  energy  of  the  group 

• Telepresence  - real  time  interaction  builds  a sense  of  social  presence  and  involvement 

• Feedback  - synchronous  systems  provide  quick  feedback  and  support  consensus  and  decision  making 

• Pacing  - synchronous  events  encourage  people  to  keep  up-to-date  and  which  helps  people  to  prioritize  their 
studies 

Two  additional  advantages  can  be  added  to  this  list: 

• Spontaneity  - synchronous  events  make  it  easy  to  add  new  ideas  to  the  conversation. 

• Familiarity  - synchronous  systems  can  simulate  a more  traditional  environment. 

Student  Perceptions  of  Synchronous  Communication 

In  an  informal  survey  of  graduate  students  enrolled  in  an  online  class,  two  questions  were  asked:  (1)  what 
form  of  Web-based  communication  do  you  feel  is  more  efficient  for  facilitating  group  discussions?  (2)  what  form  of 
Web-based  communication  do  you  prefer  in  your  online  courses?  39  students  participated  in  the  survey.  For  the  first 
question,  only  12  students  (30.8%)  responded  that  asynchronous  communication  was  more  efficient  than 
synchronous  communication  while  27  students  (69.2%)  believed  the  opposite  was  true.  Yet,  in  response  to  the 
second  question,  only  3 students  (7.7%)  responded  that  they  preferred  synchronous  communication  in  their  online 
courses.  Despite  the  perceived  efficiency  of  synchronous  communication,  a clear  majority  (69.2%)  responded  that 
they  preferred  to  communicate  asynchronously  in  their  distance  education  courses. 

Along  with  the  prior  literature,  this  internal  survey  illustrates  a trend  in  distance  education  programs  of 
emphasizing  convenience  over  efficiency.  This  dichotomy  is  understandable  when  one  considers  the  needs  of  adult 
learners.  On  the  one  hand,  they  need  and  desire  feedback  and  interaction  between  and  among  their  peers  and 
instructors.  On  the  other  hand,  they  also  need  flexibility  and  convenience  to  balance  busy  schedules  to  work  around 
professional,  family,  and  learning  responsibilities.  The  results  of  this  survey  underscore  the  need  to  find  a balance 
between  synchronous  and  asynchronous  communication  in  distance  education  programs,  exploiting  the  benefits  of 
both  while  minimizing  their  shortcomings. 

Practices 

At  the  University  of  Texas  at  Brownsville  (UTB),  our  frilly  Web-based  Educational  Technology  Program  is 
fortunate  enough  to  receive  resources  from  both  the  UTB  and  the  University  of  Texas  TeleCampus  (UTTC),  where 
most  of  our  graduate  courses  are  facilitated  through.  We  have  been  able  to  adopt  the  following  synchronous 
communication  tools: 

1.  Audiobridge  - an  audio  conferencing  service  provided  by  the  UTTC,  where  students  call  in  to  participate 

2.  BlackBoard  - a course  management  system  available  through  both  UTB  and  UTTC,  where  students  type  to 
communicate 

3.  Skype  + Festoon  - a voice-over-IP  Internet  phone,  where  students  take  part  in  a voice  (or  text)  chat  and 
share  files  at  no  cost 

4.  Macromedia  Flash  Communication  Server  - the  “engine”  of  Macromedia  Breeze,  where  learners  enjoy 
video/ audio/text  conferencing 

5.  Florizon  Wimba  LiveClassroom  - an  advanced  virtual  classroom  that  has  more  communication  features 
than  the  first  four  products 

An  audiobridge  uses  a traditional  telephone  land  line  to  host  a multiluser  conversation.  Currently,  this 
service  is  offered  by  the  UTTC  to  its  constituencies  almost  24/7.  A conference  call  initiator,  who  is  usually  the 
instructor,  schedules  a conference  at  least  48  hours  ago  by  simply  submitting  an  email  or  telephone  request  to  the 
UTTC  (see  Step  1 in  Figure  1).  As  soon  as  the  request  is  fully  processed,  a phone  number  is  then  generated  and 
emailed  to  the  instructor.  The  instructor  then  forwards  the  email  to  the  class  along  with  the  instructions.  From  the 
instructor’s  perspective,  there  is  one  major  drawback  — such  service  does  incur  some  long  distance  call  fees  to 
students. 
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2.  Establish 


Figure  1.  The  procedural  map  of  Audiobridge. 

Blackboard  is  a popular  vendor’s  product  that  features  text  conferencing  (see  Figure  2).  Instructors  can 
create  as  many  chat  rooms  as  needed  to  facilitate  the  text  chat  sessions  on  the  class  and  group  levels.  The  archive 
feature  allows  for  future  references  and  clarifications.  Blackboard’s  licensing  fee  is  increasingly  becoming  a concern 
to  many  university  administrations  as  hosting  costs  continue  to  rise  unabated. 
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Figure  2.  A screen  shot  of  Blackboard  Collaboration  (a.k.a.  Chat). 
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Recently  bought  by  eBay,  Skype  is  a voice-over-IP  voice  messaging  system,  and  it  uses  the  peer-to-peer 
(P2P)  technology.  P2P  technology  can  potentially  provide  vulnerability  to  network  security,  and  is  arguably  the 
main  reason  why  P2P  and  related  applications  are  not  so  popular  among  most  campuses  from  their  administrators’ 
viewpoint.  Skype  offers  free  PC  to  PC  calls  for  up  to  five  people  (including  the  conference  call  initiator/host). 
Festoon  works  as  a video  plug-in  program  of  Skype  and  claims  that  it  can  serve  up  to  200  users  in  one  video  call 
(see  http://www.santacmznetworks.com/products  Festoon.html).  Once  Festoon  is  installed,  it  shows  on  the  top  right 
corner  of  the  Skype  window  (see  Figure  3). 


Figure  3.  A screen  shot  of  Skype  with  Festoon  activated. 

Macromedia  Flash  Communication  Server,  recently  renamed  Macromedia  Flash  Media  Server,  is  a 
software  product  that  is  the  “engine”  of  its  famous  sister  product,  Macromedia  Breeze  (now  Acrobat  Connect 
Professional),  a Web  communication  (management)  system.  For  under-funded  institutions,  Macromedia  Flash 
Media  Server  may  be  a cost-friendly  option.  There  are  websites,  such  as  http://echo.ryerson.ca/index.html,  which 
share  scripts  for  Macromedia  Flash  Media  Server  users.  At  any  rate,  potential  users  must  be  knowledgeable  about 
ActionScript,  a programming  language  specifically  for  Flash.  Figure  4 below  presents  an  interface  that  includes 
areas  for  (from  the  top)  log  in,  PowerPoint  slides,  text  chat,  and  video  (on  the  right),  as  well  as  an  audio  platform. 
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Figure  4.  A screen  shot  of  Macromedia  Flash  Communication  (Media)  Server. 

Horizon  Wimba  Live  Classroom,  another  vendor’s  product,  can  provide  nearly  all  kinds  of  the 
aforementioned  interactions.  Yet,  this  “Rolls-Royce”  type  of  conference/communication  management  system  does 
not  come  with  a cheap  price  tag  (e.g.,  licensing  fee).  Just  like  Macromedia  Flash  Communication  Server  and  Skype, 
Horizon  Wimba  Live  Classroom’s  end-users  (i.e.,  the  instructor/presenter  and  the  students/participants)  are  advised 
to  adopt  a headset  with  a microphone  on  it  and  to  subscribe  to  a wired  (as  opposed  to  wireless)  high  bandwidth 
Internet  service  for  video/audio  quality  purposes.  These  requirements  add  expenses  on  the  end  user  side.  An  up  side 
is  Wimba’s  user-friendly  interface  and  the  fact  that  Wimba  fits  seamlessly  into  Blackboard.  Figure  5 below  shows  a 
screen  shot  of  the  Horizon  Wimba  Live  Classroom  integrated  into  Blackboard. 
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Figure  5.  A screen  shot  of  Horizon  Wimba  Live  Classroom,  embedded  in  Blackboard. 

Recommendations  for  the  Practitioners 

Effectively  integrating  synchronous  communication  into  an  asynchronous  online  environment  requires 
structure  and  effective  conference  moderation  of  the  discussions.  The  following  tips  are  recommended: 

• Decide  what  your  objectives  are  for  using  synchronous  communication. 

• Outline  the  rules  for  participation  in  your  syllabus. 

• Be  organized.  Successful  sessions  need  to  be  planned  carefully.  Prepare  a focused  topic  in  advance  for  each 
session. 

• Hold  an  orientation  at  the  beginning  of  the  semester,  where  protocols  and  features  are  demonstrated  and 
disseminated. 

• Have  a “Plan  B,”  in  the  event  of  technical  problems. 

• Allow  time  for  questions. 

• Inform  learners  of  your  expectations. 

• Schedule  formal  or  informal  practice  sessions,  of  which  “a  feature  of  the  day”  is  re-introduced. 

• Separate  group  deliberation  rooms  (e.g.,  Group  l”Chowhounds”)  from  the  class  discussion  room  (e.g., 
Town  Hall). 

• Use  audiovisuals  when  appropriate  in  the  synchronous  sessions. 

• Be  flexible.  Any  live  conversation  can  potentially  metamorphose  into  the  “teachable  moment.”  Be  ready  to 
move  in  whatever  direction  the  discussion  may  lead,  but  be  prepared  to  redirect  the  conversation  if  it  goes 
off-topic. 

• Be  aware  of  students  who  do  not  to  participate.  Determine  if  their  nonparticipation  is  due  to  a technical  or 
skill  problem,  or  a certain  reluctance  to  engage  in  classroom  dialogue. 

Summary 

Clearly,  in  the  evolving  world  of  distance  education,  synchronous  communication  has  a significant  role  to 
play.  While  asynchronous  delivery  of  instruction  continues  to  play  an  important  part  in  providing  instruction  that  is 
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free  of  time,  place  and  scheduling,  the  synchronous  approach  adds  immediacy,  live  interaction  and  personal  contact. 
This  type  of  interaction  is  endorsed  by  Wang  and  Newlin  (as  cited  in  Pan  & Sullivan,  2005,  p.  27)  who  argue, 
“...asynchronous  tools  are  viewed  as  the  backbone  and  muscle  for  the  subject  content,  whereas  synchronous 
media. . .are  the  heart  and  hustle  of  online  courses.”  Both  forms  of  instructional  strategies  can  peacefully  co-exist  if 
care  is  taken  to  appreciate  their  relative  strengths  and  weaknesses.  As  with  all  technology  and  the  technological 
symbiosis,  care  should  be  taken  to  match  the  technology  to  the  learning  objectives,  the  instructional  activity  design, 
and  attributes  of  the  students. 
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The  Effectiveness  of  different  podcast  formats,  usage  patterns  and  styles 
of  podcasting  in  delivery  of  curriculum  materials 
Comparative  Survey/Test  Snapshot  of  a Trend  or  Opportunity: 
Podcasting  Mobile  Media  Delivery  to  the  NetGen 

By  Craig  Crowe,  M.A. 

University  of  Houston 


Description 

The  School  of  Communication  at  the  University  of  Houston  is  studying  the  benefits  of  podcasting  in  the 
distribution  of  specific  curricular  content.  Various  entities  within  study  group  are  investigating  podcasting 
and  its  potential  place  in  Academia,  usage  and  impact  in  higher  education.  This  paper  will  attempt  to 
analyze  student  use  of  podcasting  as  a delivery  medium  for  supplemental  materials,  preferred  type  of 
podcast,  and  student  perceptions  in  benefit  and  learning. 

Abstract 

We  are  already  reading  about  and  experiencing  the  use  of  podcasting  as  a tool  in  the  classroom.  However, 
are  the  students  benefiting?  Is  it  just  a fad?  Is  this  really  some  form  of  flash  learning?  The  uses  of  the  iPod 
or  other  mobile  media  device  are  some  of  tools  by  what  has  been  termed  the  “NetGen”  or  iGeneration 
(Wikipedia)  group  of  students  today  use.  Since  its  introduction  in  2004  the  iPod  has  seen  its  usage  rise  to 
the  top  of  the  list  of  all  other  mobile  media  delivery  devices.  It  has  also  taken  a foothold  and  rise  in  the 
academic  field  as  a possible  media  delivery  portal  or  path  to  the  students  who  belong  to  the  iGeneration  or 
that  group  of  students  who  consume  and  use  the  Internet  and  mobile  media  and  their  devices  on  a daily 
basis.  Podcasting  delivers  an  electronic  message  that  can  be  downloaded  and  playbacked  on  a personal, 
mobile  media  device  or  MMD.  More  and  more,  educators  are  experimenting  with  podcasting;  and  it  is  a 
means  to  an  end  for  educational  content  delivery. 

As  part  of  an  investigation  team,  we  plan  to  see  if  we  can  take  a snapshot  of  how  students  enrolled  in  the 
large  lecture  classes  might  benefit  from  supplemental  materials  delivered  via  podcasting.  We  would  also 
like  to  examine  the  types  of  podcasts  used,  where  and  how  the  podcasts  are  viewed,  and  if  these  podcasts 
beneficial.  There  is  also  a mobility  issue:  Might  the  students  use  the  material  via  a computer  connected  to 
the  Internet  using  an  aggregator  like  iTunes,  a portable  MP3  player  iPod? 

The  designs  of  the  investigation  are  to  gather  data  based  on  surveys  and  test  scores,  and  compare  that  data 
to  a control  group.  At  the  School  of  Communication  the  course  Introduction  to  Media  Production  will  be 
used.  The  textbooks  and  lecture  materials  such  as  PowerPoint  presentations  will  be  the  same.  Each  class 
will  be  taught  in  the  face-to-face  lecture  style.  The  use  of  in-class  technology  will  consist  of  a projection 
system  utilizing  PowerPoint  presentations  as  support  to  the  in  class  lectures.  These  PowerPoint 
presentations  will  be  posted  in  WebCT  so  students  may  retrieve  them  from  this  web-based  tool.  The  same 
multiple-choice  tests  and  exam  dates  will  be  assigned.  One  class  will  have  the  opportunity  to  download  the 
podcasts  and  use  them,  the  other  will  not.  It  has  been  accepted  and  even  expected  that  the  class  where  the 
podcasts  will  not  be  made  available  could  find  access  to  them. 

Each  format  will  also  be  different  in  its  design.  The  audio  version  will  be  the  actual  lecture  delivered  that 
week.  The  podcast  can  be  downloaded  to  most  mobile  media  devices.  The  Enhanced  Podcast  will  include 
photographs  of  items  on  the  PowerPoint  presentations.  They  are  not  the  lectures  but  talking  points,  which 
have  limited  distribution  base  on  mobile  devices.  The  video  cast  will  be  short;  it  will  be  between  1 to  5 
minute  vignettes  from  topics  in  the  textbook.  These  topics  will  be  included  on  the  tests,  and  students  are 
expected  to  understand  in  order  to  lay  a solid  foundation  for  their  education  at  the  School  of 
Communication.  These  video  podcasts  will  also  serve  a training  opportunity  for  the  School  of 


73 


Communication  Media  Production  Curriculum.  The  instructor  using  the  podcasts  will  be  the  designer  of  the 
content,  voice  narrations  and  part  of  the  videos  too. 


Podcasts  can  be  delivered  in  three  formats,  audio  only,  still  images  with  audio  the  enhanced  podcast  or  foil 
motion  video  with  audio  in  near  DVD  quality.  The  class  will  be  presented  all  three  formats.  Data  will  be 
on: 


• Which  format  do  students  prefer,  audio  only,  enhanced  (stills  with  audio),  video,  or  all? 

• Which  platform  do  they  prefer  to  access  Podcasting,  iPod,  computer,  or  other  media 
device? 

• Where,  when,  how  often  do  students  access  podcasting? 

• What  is  the  students’  feedback  on  the  usefulness  of  Podcasting. 

Since  not  all  students  will  have  iPods  or  media  players  that  can  playback  these  podcasts,  the  students  have 
an  opportunity  to  still  use  them  as  long  as  they  have  access  to  computer  with  internet  connection.  These 
students  will  be  able  to  use  iTunes  on  their  computer,  the  aggregator,  to  access  the  podcasts  or  save  then  to 
harddrives  or  burn  to  CD.  Survey  data  will  be  gathered  on  this  usage  as  well. 

The  data  will  be  gathered  in  simple  surveys  distributed  to  students  through  out  the  semester.  The  test  scores 
will  also  be  reviewed  as  possible  indicators  to  what  the  podcast  may  if  any.  The  data  will  be  compared  to 
another  class,  which  will  be  taught  similar  curriculum,  and  the  lectures  will  use  WebCT  as  supplement  but 
without  the  access  to  the  podcasts. 

The  class,  which  is  not  instructed  to  the  podcast,  is  believed  to  most  likely  gain  access  to  the  podcasts  and 
use  them.  An  anonyms  survey  will  be  given  to  this  class  as  an  attempt  to  seek  the  data  on  usage  and 
possible  effects. 

This  investigation  is  just  the  beginning  of  trying  to  track  patterns  of  usage  in  what  some  view  as  a trend  in 
the  educational  field.  Podcasting  could  be  another  tool  to  further  education,  and  reach  a generation  of 
students  who  can  learn  in  a mobile  environment.  It  is  whether  content  delivered  is  retained  and  used  by 
these  students  along  with  the  how  when  and  where  that  can  better  facilitate  the  message  delivered  by  the 
educator. 

Survey  Instrument 

The  University  of  Houston  Podcasting  Survey  was  conducted  in  the  Spring  semester  of  2006.  One  hundred 
sixty  one  (161)  students  participated  from  4 classes  in  2 courses,  which  the  data  was  gathered  from. 

Students  were  enrolled  in  Communications  3353  Web  Technologies  where  one  class  was  conducted  on  line 
and  the  other  was  taught  face  to  face.  The  second  course  was  Communications  2320  Introduction  to  Media 
Production  with  both  classes  taught  face  to  face  and  both  larger  lecture  formats.  One  section  of  2320  had 
access  to  the  podcasting  material  form  the  beginning  of  ht  semester,  while  the  other  section  introduced 
podcasting  at  midterm 

Students  where  given  same  Survey  Instrument  at  the  end  of  the  Spring  Semester  2006.  The  Survey  consists 
of  7 Sections  that  gathered  data  on  Demographic  information.  What  type  of  iPod  used.  What  type  of  MP3 
player  used,  Use  while  involved  in  other  activities,  Podcasting  course  material.  Podcasting  and  learning 
styles  and  study  habits,  and  lastly  Podcasting  and  learning  effectiveness. 

Demographic  Information. 

The  study’s  sample  population  was  predominately  in  the  20-25  year-old  are  range,  which  places  them  in  the 
iGeneration  group.  This  generational  group  tends  to  be  raised  using  the  Internet,  MySpace,  and  iPod 
(Wikipidia).  Over  80%  of  the  sample  population  had  high-speed  Internet  connections  with  only  1 1%  on 
slower  dial  up  lines.  This  group  is  much  more  adapted  to  quicker  access  to  Internet  materials  as  the  survey 
supports.  They  have  been  using  the  Internet  for  much  longer  as  well,  with  over  60%  of  the  respondents 
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having  over  8 years  of  usage.  Format  of  the  course  was  important  as  well,  traditional  face-to-face,  no  online 
usage,  traditional  face-to-face,  some  online  usage,  hybrid/blended  class  (50%  or  more  online)  and  fully  on 
line. 

What  kind  of  internet  connection(s)  do  you  have? 
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What  is  the  format  of  the  course  you  are  currently  taking? 


Mobile  Media  Device  (MMD)  Ownership 


Ownership  of  mobile  media  devices  (MMD)  like  iPod  models  and  other  MP3  devices  along  with  the 
knowledge  of  podcasting  was  gathered.  In  one  of  the  Web  Technologies  courses  the  video  iPod  was 
distributed  to  the  students,  all  other  surveyed  courses  the  student  had  to  have  their  own.  Student  could  still 
access  the  Podcasts  using  their  computers  if  they  did  not  have  an  MMD  or  if  they  choose  too.  Further 
questions  involved  usage  of  the  devices,  and  preferred  use  by  the  Student. 
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Did  you  know  what  a podcast  was 
before  you  took  this  course? 


□ Yes 

□ No 


Did  you  ever  listen  to  a podcast  before 
you  took  this  course? 


□ Yes 

□ No 


As  seen  the  majority  of  the  student  population,  over  57%  knew  what  a Podcast  was  before  the  survey 
semester,  but  only  a little  over  31%  had  actually  listened  to  a Podcast.  This  would  give  the  indication  that 
the  students  where  aware  of  the  technology,  but  it  was  still  under  utilized. 

The  term  Podcasting  can  seem  to  be  exclusionary,  being  so  closely  associated  with  various  iPods  models 
and  the  Apple  computer  product  line.  Podcasts  can  be  view  on  PC  computers  as  well  and  Podcasts  can  be 
downloaded  to  other  non-iPod  MP3  players.  Students  were  allowed  to  use  the  MMD  of  their  choice. 
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Indicate  which  model(s)  of  iPod(s)  you  currently  own 


iPod 

How  many  hours  do  you  use  your  iPod  a week? 


hours  a aweek  aweek  a week  aweek  week  hours 

week 


When  do  you  prefer  to  use  your  iPod? 


What  activities  do  you  do  most  with  your  iPod? 
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How  many  hours  do  you  use  your  MP3  player  a week? 


week 


When  do  you  prefer  to  use  your  MP3  player? 


car  jogging  location 

What  activities  do  you  do  most  with  your  MP3  player? 


materials 


news 


Here  the  data  gathered  supports  whether  the  student  has  an  iPod  or  other  MP3  device.  The  usage  patterns 
are  similar  in  hours  used,  when  used,  and  with  music  being  the  most  reported  activity.  This  is  not  surprising 
as  these  devices  are  targeted  as  personal  music  players,  yet  one  can  see  that  if  offered  the  student  would 
also  access  course  materials  if  made  available. 

Student  Usage  and  Formats 

The  survey  found  a wide  range  of  reasons  students  chose  to  access  course  material  via  the  podcast.  There 
was  no  favorite  delivery  style;  audio  cast,  enhanced  cast  or  video  cast.  Yet  when  further  questioned,  over 
56%  of  the  students  stated  a preference  for  video  casts  as  it  most  suited  their  learning  style. 
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Majority  of  the  student's  surveyed  are  using  their  mobile  media  devices  less  than  5 hours  to  access  course 
materials,  this  data  could  help  shape,  how  the  podcast  formats  shoidd  be  in  content  and  duration.  Possibly 
short  duration  messages,  or  burst  messaging,  would  benefit  in  the  restrictive  technological  confines  of  the 
mobile  media  devices.  The  survey  supported  this  concept  as  shorter  messages  were  constructed  for  the 
enhanced  or  still  image  format  and  the  video  format  as  well.  Audio  casts  where  usually  longer  as  they 
where  full  lectures  by  nature,  but  these  to  can  be  packaged  in  chapter  format  allowing  the  student  to  skip 
through  the  lecture  to  better  benefit  their  learning. 

As  a new  and  developing  method  of  course  delivery,  the  responsibility  well  be  placed  on  the  educator  to 
present  different  possible  applications  of  the  technology  to  the  student  population.  Presenting  course 
materials  through  a different  channel  will  not  be  enough.  Integrating  this  technology  in  to  the  daily  learning 
experience  will  be  vital.  Our  survey  supports  this. 


The  length  of  the  podcasts  were: 


□ Too  long 

□ Just  right 

□ Too  short 
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Please  rate  the  following  delivery  quality  of  podcast  course  materials. 

(Still  Images) 


□ Excellent 

□ Good 


□ Poor 
B Bad 

□ N''A 


Please  rate  the  following  delivery  quality  of  podcast  course  materials. 

(Video) 


□ Excellent 


□ Good 

□ Poor 

□ Bad 

□ N/A 

□ Missing 
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Effectiveness 


The  use  of  Podcasting  by  the  Communication  Department  was  well  received  by  the  student  population. 
Most  found  the  use  of  Podcasts  improved  their  learning  outcome  and  made  their  perceived  learning  more 
enjoyable.  The  students  report  that  they  had  a better  understanding  of  the  course  materials  and  that  a sense 
of  freedom  was  afforded  them  by  allowing  review  of  materials  whenever  they  want.  The  concept  of  “Time- 
Shifting  or  the  ability  to  listen  material  when  we  choose”  (Duncan  and  Fox,  2005)  is  a key  component  to 
the  control  the  students  will  feel  over  their  own  educational  attainment.  Still  the  students  indicated  their 
preference  for  a combination  of  podcasts  and  traditional  face-to-face  content  delivery. 

The  addition  of  podcast  material  was  helpful  to  the  understanding  of 

course  content. 


Agree  Disagree 


Listening  to  podcasts  helped  me  better  retain  the  content  information. 


I prefer  the  combination  of  iPod/MP3  player  course  delivery  with  the 
face-to-face  classroom  experience. 


81 


Which  of  the  following  features  of  podcasts  did  you  find  most  valuable? 
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C 

a) 

o 

i— 

a> 

0. 


60- 

50- 

40- 

30- 

20- 

10- 


0' 


1 

Ability  to  review  materials 
wherever  I wanted. 


1 

Ability  to  review  materials 
whenever  I wanted. 


1 

Ability  to  review  materials  at  my 
own  pace. 


The  podcast  delivery  format  enhanced  my  learning  experiences. 


Strongly  Agree  Agree 


N/A 


Disagree 


Strongly 

Disagree 


Podcast  delivery  format  made  learning  more  enjoyable. 


50- 
c 4°- 

8  I *  3°- 
o 20- 
10- 
0-- 


I I I I I 

Strongly  Agree  Agree  N/A  Disagree  Strongly 

Disagree 
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Conclusion 


The  School  of  Communication  at  the  University  of  Houston  has  begun  to  investigate  the  various  types, 
styles  and  effects  of  Podcasting.  This  paper  is  just  a snapshot  of  an  on  going  process.  Further  research  will 
be  conducted  expanding  the  course  offerings  and  widening  the  student  population  sample. 

Our  findings  support  that  the  “students  are  characterized  as  social,  highly  competent  multitaskers,  who 
expect  immediate  results  and  feedback  and  seek  stimulation  and  interaction”  (Tucker  2006).  The  computer 
literate,  iGeneration,  is  aware  of  Podcasting  technology.  Receptive  to  its  alternative  delivery  mechanisms 
for  course  materials  can  be  used  to  meet  their  lifestyles  and  educational  attainment. 
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Strengthening  Connections  Between  Institutional  Staff  Members  and  Clients: 
Design,  Development,  and  Implementation  of  a Sign  Language  Learning  Tool 
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Abstract 


Among  its  many  uses,  sign  language  facilitates  communication  by  individuals  with  mental 
retardation,  autism,  and  other  similar  conditions.  Institutional  staff  members  who  work  with  these 
individuals  often  need  to  learn  sign  language  to  communicate  with  their  clients.  This  paper 
describes  a new  approach  to  helping  hearing  people  practice  sign  recognition  and  discusses  a new 
sign  language  practice  tool  that  incorporates  this  approach.  The  tool  is  currently  being  used  by  a 
population  of  institutional  staff  members. 

Descriptors:  Instructional  Development,  Training,  Sign  Language 


Background 

A major  difference  between  sign  languages  and  spoken  languages  is  in  the  faculties  and  senses  used.  The 
means  of  output/expression  for  a sign  language  is  primarily  gesture,  whereas  output/expression  of  a spoken  language 
is  primarily  by  speech  itself.  Input/reception  of  a sign  language  is  accomplished  by  means  of  vision  or  touch,  as 
opposed  to  that  of  a spoken  language,  which  is  primarily  through  hearing  (Armstrong,  Stokoe,  & Wilcox,  1995). 

Prior  to  the  1960’s  all  linguistic  study  was  of  spoken  language  (Klima  & Bellugi,  1979).  Sign  languages 
were  mistakenly  considered  a form  of  pantomime  with  none  of  the  characteristics  of  language.  At  that  time,  it  was 
believed  that  signs  were  the  smallest  components  of  meaning  and  that  there  were  no  separable  descriptors  of  a sign 
that  would  convey  meaning.  Stokoe,  Liddell,  and  Johnson  (Stokoe,  Casterline,  & Croneberg,  1976;  Schein  & 
Stewart,  1995;  Valli  & Lucas,  2000)  conducted  pioneering  analyses  of  sign  language,  specifically  American  Sign 
Language  (ASL)  the  sign  language  of  the  Deaf  community  in  North  America,  that  demonstrated  that  signs  are 
composed  of  meaningful  sub-units  that  can  be  used  to  distinguish  one  sign  from  another.  Those  sub-units, 
handshape,  location,  movement,  and  palm  orientation,  are  comparable  to  the  phonemes  of  spoken  language. 

While  there  have  been  several  attempts  to  develop  a written  form  of  sign  language,  none  are  generally 
accepted  to  the  extent  that  the  English  alphabet  is  accepted  as  a method  of  writing  English  (Schein  & Stewart,  1995; 
Wilcox  & Wilcox,  1997).  As  a result,  English  glosses  are  frequently  used  to  provide  documentation  of  sign 
language.  Hearing  students  learning  sign  language  typically  use  glosses  to  help  them  understand  and  document  the 
meaning  of  the  signs.  Sign  phonemes  and  glosses  are  commonly  used  by  sign  learners  to  document  signs,  but 
neither  of  these  descriptive  techniques  replace  the  need  to  see  demonstrations  of  the  signs  themselves. 

Sign  language  is  often  used  to  facilitate  communication  for  individuals  with  mental  retardation,  autism,  or 
other  conditions  that  require  that  they  live  in  a supervised  setting.  Use  of  sign  language  by  residents  and  staff  can 
help  channel  communication,  reduce  frustration,  and  facilitate  positive  social  behavior  (Kiernan,  Reid,  and  Jones, 
1982;  Musselwhite  & St.  Louis,  1988;  Rittenhouse  & Myers,  1982).  The  sign  language  used  in  this  setting  usually 
takes  the  form  of  a local  variation,  composed  of  a limited  vocabulary  of  signs  that  can  be  learned  by  the  residents 
and  used  to  convey  basic  needs,  thoughts,  and  feelings.  Local  variation  signs  are  usually  borrowed  from  natural  sign 
languages  such  as  American  Sign  Language  or  an  educational  sign  system  and  may  be  customized  to  provide 
memory  cues  (Musselwhite  & St.  Louis,  1988;  Rittenhouse  & Myers,  1982).  It  is  extremely  important  that 
caregivers  develop  communications  systems  with  clients  (Musselwhite  & St.  Louis,  1988).  Successful 
establishment  of  a communication  partner  leads  to  much  greater  success  in  language  learning  for  clients. 

Sign  language  is  a visual-spatial  language,  in  that  production  takes  place  in  4-dimensional  space  (i.e.  width, 
height,  depth,  time).  The  shift  in  language  modality  from  aural-oral  to  visual-gestural  presents  particular  difficulties 
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for  the  hearing  signer.  They  must  develop  both  receptive  (able  to  view  and  comprehend  signing)  and  expressive 
(able  to  produce  signs)  sign  skills.  Hearing  signers  must  be  able  to  view  others  using  the  language  and  be  able  to 
identify  changes  in  form  and  gesture  instead  of  relying  on  already  well-tuned  aural  skills  to  pick  up  changes  in  tone, 
pitch,  and  articulation. 

Hearing  people,  when  learning  sign  language,  have  much  more  difficulty  recognizing  signs  than  producing 
them  (Poor,  personal  communication,  July  17,  2003;  Tennant  & Brown,  1998).  This  means  that  opportunities  to 
practice  and  reinforce  recognition  skills  are  particularly  important. 

Another  difficulty  novice  sign  language  learners  face  is  the  need  to  mentally  rotate  the  person  who  is 
signing.  Since  most  sign  conversations  take  place  with  one  signer  facing  another,  the  mental  model  of  sign 
production  is  at  a 180-degree  rotation  from  the  mental  model  of  sign  reception.  Bellugi,  Klima,  and  Siple  (1975) 
describe  the  required  rotation  of  mental  models  as  transformation.  It  complicates  and  slows  sign  reception  further 
(Schein  & Stewart,  1995).  A sign  learning  approach  should  provide  a variety  of  angles  of  demonstration  to  help 
learners  internalize  the  two  different  mental  models  for  reception  and  expression. 

In  order  to  further  support  the  change  in  modality  and  interpretation  of  multiple  cues,  the  instructional 
design  should  include  sign  demonstrations  that  are  as  close  to  real-life  as  possible.  Signs  should  be  demonstrated 
realistically,  with  the  option  to  modify  the  viewpoint,  in  order  to  allow  the  student  to  review  all  of  the  relevant 
phonemes,  to  view  multiple  visual  cues  at  once,  and  to  adjust  to  the  two  mental  models. 

Some  elements  of  sign  language  actually  help  the  sign  student.  One  of  the  most  important  is  the  use  of 
handshapes  to  form  signs.  Most  of  the  required  handshapes  are  also  letter  signs  of  the  American  Manual  Alphabet 
or  variations  on  them.  In  most  cases,  the  new  signer  learns  to  fingerspell  before  learning  actual  signs.  This  exercise 
helps  the  signer  learn  most  of  the  handshapes  required  (Schein  & Stewart,  1995).  It  also  means  that  handshape  is 
the  first  phoneme  learned.  Wilcox,  Scheibman,  Wood,  Cokely,  and  Stokoe  (1994)  found  that  the  handshape  is  the 
phoneme  most  likely  to  be  perceived  independently  of  other  phonemes  when  identifying  a sign.  In  many  cases, 
local  variations  of  sign  language  in  institutional  settings  use  letter-cued  versions  of  an  ASL  sign,  to  help  hearing 
staff  remember  signs. 

Most  members  of  the  target  group  are  adult  learners.  This  is  an  important  consideration  because  adult 
learners’  needs  and  skills  are  different  from  those  of  younger  learners  (Knowles,  Holton,  and  Swanson,  1998). 

Time  constraints  further  complicate  the  task  of  learning  sign  language.  Institutional  staff  members  are  busy 
and  have  little  free  time  during  a typical  workday.  Funding  reductions  and  hiring  difficulties  contribute  to  the  need 
for  institutional  staff  to  multi-task,  reducing  free  time  further.  In  order  to  be  useful  to  staff  members,  training 
programs  and  materials  must  be  flexible  in  terms  of  resources  and  time  required  (Fitzgerald,  et  al.,  1984).  Staff 
members  need  to  be  able  to  develop  signing  skill  in  as  little  time  as  possible.  This  can  be  accomplished,  in  part,  by 
dividing  the  signs  into  small  subsets  that  can  be  studied  in  a minimal  time  frame.  In  addition,  staff  members  should 
be  able  to  focus  on  signs  that  are  a part  of  the  local  variation  instead  of  learning  many  signs  that  are  not  used  or  that 
are  formed  differently  from  the  local  variation.  Thus,  a sign  learning  tool  should  present  a set  of  signs  customized 
for  the  institution’s  local  variation  and  that  are  segmented  into  manageable  subsets. 

Any  learning  approach  should  support  flexibility  for  learners  who  may  need  a varying  number  of  sessions 
to  master  a particular  set  of  signs.  This  flexibility  will  encourage  learners  to  use  the  approach  because  they  do  not 
need  to  start  at  or  reach  the  same  point  as  others  who  are  also  using  the  same  approach  (Bogue,  1989). 

One  recommended  strategy  for  teaching  psychomotor  procedures,  such  as  sign  language,  includes 
modeling  or  demonstrating  the  task,  encouraging  the  student  to  form  mental  images  of  the  procedure,  and  practicing 
(Morrison,  Ross,  & Kemp,  2003).  As  noted  above,  reliance  on  vision  and  lack  of  a written  form  make  sign  language 
learning  a very  different  experience  from  spoken  language  learning.  In  order  to  successfully  support  sign  language 
acquisition,  instructional  tools  should  include,  at  a minimum, 

• demonstration  - modeling  signs  consistently  in  4 dimensions  (width,  height,  depth,  time)  with  unlimited 

repetition  from  more  than  one  angle  of  observation 
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access  - allowing  students  to  search  for  sign  demonstrations  for  reference  and  follow-up  based  on  an 
observable  linguistic  characteristic  of  the  sign. 


Previous  Approaches  to  Sign  Language  Learning 

Until  recently,  most  sign  language  instruction  had  been  conducted  in  person.  The  most  predominant 
techniques  were  face-to-face  classroom  instruction  and  practice  with  other  signers  and  sign  learners.  However, 
instructors  and  other  signers  are  not  always  available  to  demonstrate  signs  for  students  and  the  demonstrations  may 
vary  due  to  fatigue,  signing  idiosyncrasies,  and  other  considerations. 

Sign  learners  have  also  used  textbooks  and  other  printed  references  to  reinforce  learning  in  study  sessions 
outside  the  classroom.  Students  would  study  static  photographs  or  drawings  of  sign  production,  usually  labeled  with 
an  English  gloss.  As  noted  above,  sign  language  conversations  occur  in  four  dimensions,  and  to  be  most  effective, 
demonstrations  of  sign  language  should  also  occur  in  four  dimensions.  This  means  that  instructional  tools  that  do 
not  support  depth  or  time  such  as  printed  material  or  still  images  will  not  be  as  effective  as  those  that  do. 

There  are  also  a wide  variety  of  videotape-  and  computer/CD-based  products,  from  narrated  demonstrations 
of  individual  signs  to  videotaped  instructional  programs  that  are  intended  to  take  the  learner  through  an  entire 
introductory  course  in  sign  language.  However,  videotape-based  products  are  limited  in  presentation  quality  and 
access,  in  large  part  because  videotape  is  a sequential  medium.  While  videotape-based  materials  can  be  rewound  to 
show  signs  multiple  times,  the  action  of  working  with  the  videotape  player  to  access  specific  video  segments 
distracts  the  student  from  the  learning  process.  Existing  CD/computer-based  sign  language  products  are  limited  in 
their  ability  to  present  demonstrations  at  various  angles,  quiz  content,  and  sign  vocabulary.  Some  products  use 
recorded  videos  of  human  signers,  while  others  present  animated  signers. 


A New  Approach 

The  PAULA  sign  language  tutor  was  originally  developed  for  a specific  population  of  sign  language 
learners:  institutional  staff  at  the  Jack  Mabley  Developmental  Center,  a state-operated  residential  facility  located  in 
Dixon,  Illinois.  Information  gathering  to  support  this  project  focused  on  the  needs  of  Mabley  Center  staff  as  they 
learn  the  signs  of  a local  variation  used  at  the  Mabley  Center.  Local  variations  are  sign  languages  developed  to 
serve  a particular  population  within  a closed  community,  such  as  a hospital  or  care  center.  They  have  a limited 
vocabulary  that  is  based  on  the  needs  of  the  population.  Local  variation  signs  are  usually  borrowed  from  natural 
sign  languages  such  as  American  Sign  Language  or  an  Educational  Sign  System  and  may  be  customized  to  provide 
memory  cues  (Musselwhite  & St.  Louis,  1988;  Rittenhouse  & Myers,  1982). 

The  PAULA  sign  language  tutor  presents  sign  demonstrations  via  a series  of  3-D  graphical  animations. 
Animations  of  three  presentation  angles,  front,  side  and  top,  are  available  for  each  of  170  signs  that  have  been 
transcribed  for  the  tutor.  The  sign  transcription  process  records  both  geometric  and  linguistic  information,  such  as 
handshape,  for  the  sign.  The  geometric  information  is  used  to  generate  3-D  graphical  animations  of  the  sign,  and  the 
same  information  is  available  in  a database  that  can  provide  descriptions  of  the  associated  sign  characteristics  for 
the  learner  The  transcription  approach  and  the  resulting  database/animation  synchronization  eliminate  the  need  for 
the  tedious  and  error-prone  post  process  of  manual  annotation  of  sign  presentations  used  in  many  other  products. 
Review  of  the  resulting  sign  animations  for  visual  accuracy  provides  a double  check  of  the  information.  The 
database  design(Furst,  et  al.,  2000;  Wolfe,  et  ah,  1999)  is  based  on  the  concept  that  the  most  important  building 
block  of  a sign  is  the  handshape,  which  is  the  most  recognizable  and  memorable  aspect  of  a sign  (Wilcox  & 
Scheibman,  n.d.). 

The  author  interviewed  the  Mabley  Center  staff  to  understand  the  needs  that  the  tutor  was  to  meet.  This 
included  observations  of  sign  language  classes  conducted  at  the  Center. 

The  first  prototype  allowed  the  user  to  look  up  a sign  by  gloss  and  provided  a multiple-choice  sign-quiz. 
The  prototype  also  allowed  the  user  to  choose  the  angle  of  presentation  (front,  side,  top).  It  included  a multimedia 
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control  so  that  the  user  could  play  and  re -play  the  animated  sign  demonstrations.  Screen  shots  of  the  first  Sign 
Language  Learning  Tool  prototype  are  included  below  as  Figure  0. 


Figure  0.  Screens  from  First  Prototype 
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Insights  Gained  Through  User  Testing  of  the  First  Prototype 


In  September  2003,  twelve  Mabley  staff  members  reviewed  the  first  Learning  Tool  prototype.  The  staff 
members  had  a variety  of  levels  of  experience,  training,  and  sign  skill.  Each  reviewer  started  with  the  Quiz  section 
of  the  prototype.  The  reviewer  could  view  the  Demonstration  section  at  any  time.  Reviewers  were  encouraged  to 
comment  on  the  prototype  interaction.  The  author  noted  interaction  paths  and  variations,  as  well  as  reviewer 
comments.  The  Mabley  project  coordinator  remained  in  the  room  during  the  reviews  and  testing,  acting  as  a 
facilitator  during  the  review  process.  She  had  contacted  the  volunteers  and  organized  the  sessions.  All  reviewers 
felt  this  tool  could  be  useful  to  them  as  they  practice  sign  language.  All  were  very  positive  about  it  and  saw  its 
potential. 


Changes  Implemented  in  the  Second  Learning  Tool  Prototype 


User  feedback  dictated  the  development  of  additional  features,  including  an  open-ended,  fill-in-the-blank 
quiz  and  a section  that  documents  learning  strategies  recommended  by  the  sign  language  instructor.  The  second 
quiz  format  is  helpful  for  students  who  have  progressed  beyond  the  beginning  stages  of  sign  learning.  In  addition, 
the  author  simplified  the  controls  by  converting  the  view  selection  controls  from  tabs  to  radio  buttons  and  providing 
a PLAY  button  instead  of  the  multimedia  control.  The  feedback  area  size,  color,  and  position  were  modified  to 
make  it  more  easily  noticed.  The  author  implemented  keyboard  shortcuts  and  tab  stops.  The  author  installed  the 
second  Learning  Tool  prototype  at  the  Mabley  Developmental  Center  in  November  2003. 
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Screen  shots  of  the  PAULA  software  are  presented  in  Figures  1-8  below. 


Figure  2.  Sign  Look-up  by  English 
Gloss 


Figure  3.  Multiple-Choice  Sign 
Quiz 
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The  user  can  also  choose  the  angle  of  presentation  —front,  side,  top  (See  Figure  4 ) . The  side  view  is 
helpful  in  those  cases  where  a sign  includes  location  and  movement  that  are  difficult  to  perceive  from  the  front.  The 
top  view  is  beneficial  in  helping  users  internalize  the  two  different  mental  models  for  reception  and  expression. 


Figure  4.  Presentation  Angles 


The  tutor  was  enhanced  to  include  an  open-ended,  fill-in-the-blank  sign  quiz  and  a section  that  documents 
learning  strategies  recommended  by  the  sign  language  instructor.  The  second  quiz  format  was  developed  for 
students  who  have  progressed  past  the  beginning  stages  of  sign  learning.  When  taking  the  fill-in-the-blank  sign  quiz, 
the  user  keys  in  a gloss  that  is  compared  with  the  gloss  of  the  demonstrated  sign.  For  all  quiz  types,  the  user  receives 
text  feedback,  either  CORRECT  or  TRY  AGAIN  (See  Figure5a,5b). 


Figure  5a.  TRY  AGAIN 


Figure5b.  CORRECT 
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Changes  Implemented  Based  on  the  Results  of  Sign  Search  Testing 


The  author  subsequently  developed  a learning  tool  feature,  Sign  Search,  which  allowed  a user  to  search  for 
a sign  by  handshape  name  or  handshape  appearance.  The  Sign  Search  software  generates  statements,  which  query  a 
sign  language  database  created  by  the  sign  transcription  process.  (See  Figure  6a, 6b). 


Oout)<»-Cltct  • HOTdtftap*  knag*  to  Sh  iLmoUi  Stgm 

-nnnnnnnn 
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H ~ ~ Hi” 

Figure  6a.  Handshape  Choice 


Figure  6b.  Sign  Choice 


A new  version  of  the  learning  tool  software  was  created  in  response  to  the  feedback  received  during  the 
Sign  Search  tests.  It  combines  the  functionality  of  the  sign  learning  tool  and  Sign  Search.  Based  on  comments 
about  lack  of  handshape  knowledge,  the  author  added  a handshape  quiz  (See  Figure  7)  and  handshape  information 
reference.  In  addition,  the  author  developed  a quiz  results  tracking  function  that  the  user  can  turn  on  if  desired  (See 
Figure  8). 


One  of  the  most  recent  enhancements  to  the  PAULA  tutor  has  been  the  addition  of  a speed  control.  This 
control  allows  the  user  to  modify  the  pace  of  the  sign  demonstration  and  is  particularly  useful  when  a sign  is 
complex  or  includes  more  than  one  handshape. 


Summary 

The  PAULA  tutor: 

• Focuses  on  development  and  practice  of  sign  language  recognition  skills. 

• Provides  consistent  demonstrations  of  signs  at  three  different  performance  angles. 

• Presents  and  reinforces  the  sign  vocabulary  of  the  local  variation  to  be  used  by  the  learner 

• Allows  the  user  to  control  the  speed  of  sign  presentation 

• Gives  text  feedback  on  three  different  types  of  quizzes  (open-  and  closed-end  sign  and  handshape) 

• Tracks  quiz  performance  upon  user  request. 

• Allows  the  sign  language  learner  to  control  the  practice  and  feedback  process. 
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The  PAULA  tutor  has  been  packaged  and  distributed  to  Mabley  Center  staff  members  and  families  of 
Mabley  Center  residents.  User  comments  have  been  very  favorable  and  a second  distribution  was  requested.  This 
distribution  was  made  possible  by  a grant  from  the  Dixon  Area  Retarded  Citizens  group  and  was  completed  earlier 
this  year. 


The  Mabley  Center  PAULA  tutor  can  be  used  by  hearing  adults  to  learn  and  review  sign  language  at  times 
compatible  with  their  individual  schedules.  The  design  of  PAULA  is  centered  on  the  needs  of  sign  language 
learners  and  the  iterative  nature  of  the  development  cycle  has  provided  many  opportunities  for  the  sign  learners  to 
make  their  needs  known.  Mabley  Center  staff  and  residents  will  continue  to  derive  associated  benefits  from 
improved  communication  based  on  more  confidence  and  skill  in  using  sign  language. 

The  Mabley  Center  Sign  Tutor  is  also  being  used  as  the  framework  for  sign  tutors  for  other  venues, 
including  a major  Chicago  hospital. 


Future  Work 

The  author  would  like  to  develop  and  implement  enhancements  to  the  PAULA  tutor  that  include  more  precise 
feedback  based  on  the  user’s  performance  history  and  characteristics  of  sign  language.  Other  potential  development 
avenues  include  modification  of  tutor  characteristics  to  make  them  less  dependent  on  English  glosses,  (e.g. 
presenting  feedback  in  sign  language  where  appropriate)  and  work  toward  a full-featured  computer-based  sign 
language  dictionary. 

The  current  PAULA  software  supports  selection  of  only  one  handshape  per  search.  Users  could  benefit 
from  the  ability  to  input  multi-stage  queries,  gradually  narrowing  the  candidate  sign  list  by  excluding  signs  that  do 
not  meet  all  criteria.  Creation  of  this  feature  would  involve  additional  design  and  development. 

One  possible  area  of  further  inquiry  would  be  a review  of  lexicography,  especially  studies  of  dictionaries  of 
spoken  languages.  In  particular,  a review  of  look-up  strategies  could  provide  useful,  generalizable  techniques  for 
further  improvement  of  the  search  interaction,  as  well  as,  comparison  of  search  efficiency  and  effectiveness. 

The  sign  language  learning  tool  described  above  is  a great  step  forward  for  institutional  staff  members  who 
are  learning  sign  and  for  those  who  want  to  find  a sign  based  on  a visual  characteristic.  Staff  members  were  able  to 
find  signs  using  the  software;  they  wanted  to  use  it  in  the  future;  and  it  was  more  accurate  than  prior  sign  look-up 
tools.  The  future  challenges  of  PAULA  software  development  include  making  the  software  even  more  useful  and 
usable  for  a wider  audience. 
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Abstract 

In  this  study,  we  investigate  the  effects  of  conversational  agents  on  communication  and  learning  when  used 
to  assist  participants  in  developing  an  online  portfolio.  Data  from  52  participants  were  gathered  and  analyzed 
through  questionnaires,  written  reflections,  transcripts  of  student-agent  interactions,  and  focus  groups.  Data  revealed 
that  participants  communicated  with  the  agents  on  issues  ranging  from  portfolio  development  to  popular  culture. 
Although  participants  did  not  view  the  agents  as  particularly  helpful  in  completing  class  activities,  they  did  use  them 
as  social  companions  throughout  the  four-week  study.  Implications  of  the  findings  for  future  design  and  research 
include:  (a)  learner-developed  conversational  agents,  (b)  improved  “intelligence”  with  which  agents  deliver  content- 
based  knowledge,  and  (c)  further  developed  virtual  characters  that  can  meet  users’  humanistic  and  utilitarian 
expectations. 

Conversational  Agents  and  Their  Longitudinal  Affordances  on  Communication  and  Learning 

Virtual  characters  are  becoming  more  common  within  industry  and  academia  where  they  have  been  used  in 
a variety  of  domains  for  multiple  purposes,  such  as  assisting  learners  on  how  to  complete  a task  (e.g.,  Baylor  and 
Ryu,  2003)  or  to  report  sports  news  online  (e.g.,  http://sports.espn.go.com/chat/sportsnation).  In  response,  a number 
of  research  studies  have  been  conducted  on  the  use  of  virtual  characters  in  educational  environments  although  these 
have  come  to  no  consensus  as  to  whether  virtual  characters  improve  learning  and  teaching.  Baylor  (2002),  Craig, 
Gholson  and  Driscoll  (2002),  and  Mayer,  Dow  and  Mayer  (2003)  for  example,  have  argued  that  the  use  of  agents 
does  not  generally  contribute  to  improved  performance,  whereas  Moreno,  Mayer,  Spires,  and  Lester  (2001)  found 
that  students  who  interacted  with  an  agent  received  higher  scores  than  those  working  on  their  own.  Dehn  and  van 
Mulken  (2000)  and  Gulz  (2004)  examined  the  proposed  benefits  of  agent-enhanced  learning  environments  and 
found  that  the  evidence  for  integrating  agents  in  educational  settings  is  at  best  mixed.  This  lack  of  unanimity  is 
further  complicated  by  differences  in  the  design  of  agents  and  experiments  (Clark  and  Choi,  2005).  For  example. 
Cole  et  al.  (2003)  had  used  an  agent  that  was  described  as  a disembodied  head  lacking  natural  facial  expressions, 
while  Louwerse  et  al.  (2005)  used  a combination  of  animated  cartoon-like  and  human-like  characters.  Only  recently 
have  researchers  proposed  frameworks  for  uniform  research  design  so  as  to  make  meaningful  comparisons  possible 
(Clark  and  Choi,  2005;  Ryu  and  Baylor,  2005) 

The  majority  of  studies  concerned  with  virtual  characters  have  been  short-term  experimental  and  quasi- 
experimental  in  nature  (Mahmood  and  Ferneley,  2006),  attempting  to  discover  relationships  between  various  virtual 
character  features  (e.g.,  image,  voice,  animation)  and  various  variables  of  interest  to  educators  (e.g.,  meta-cognition, 
motivation,  self-efficacy).  In  this  study,  we  attempt  to  complement  the  literature  on  which  features  of  virtual 
characters  may  influence  learning  by  examining  students’  multiple  interactions  with  and  responses  to  conversational 
agents  over  time,  as  well  as  the  effects  the  students  believe  these  agents  had  on  their  learning.  An  important 
weakness  of  the  current  literature  is  that  researchers  do  not  know  what  happens  when  agents  are  integrated  in 
ecologically  valid  contexts  where  learners  have  the  option  to  return  or  not  to  return  to  agent-based  systems  (Gulz, 
2004). 

We  define  conversational  agents  as  virtual  characters  that  are  able  to  sustain  a conversational  interaction 
with  students  through  students’  textual  input.  We  use  the  term  conversational  rather  than  pedagogical  agent  because 
pedagogical  agents  mainly  seem  to  refer  to  non-interactive  virtual  characters  that  deliver  content  to  students.  In 
addition,  we  abstain  from  using  the  term  intelligent  agent  because  the  word  intelligence  signifies  a higher-order 
cognitive  ability.  Even  though  the  conversational  agents  we  employ  may  appear  to  be  intelligent,  in  actuality  they 
are  not  - the  software  is  simply  trained  to  match  comments  to  responses.  The  interaction  between  student  and  agent 
is  not  pre-determined,  but  shaped  by  both  student  comments  and  agent  responses.  For  example,  if  a user  asks  the 
agent,  “What  does  it  mean  for  a website  to  be  accessible?”  the  conversation  will  center  on  this  particular  question.  If 
the  user  then  asks  a further  question  about  a specific  aspect  of  the  agent’s  answer,  the  conversation  can  be  said  to 
have  been  influenced  by  the  agent  response  but  would  still  be  dependent  on  the  user’s  subsequent  comment.  Using 
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an  artificial  intelligence  engine  (Program  Z by  Pandorabots),  student  comments  are  analyzed  into  meaningful 
segments  and,  through  an  iterative  algorithm,  are  matched  to  responses.  The  knowledge  base  is  created  via  the  use  of 
the  Artificial  Intelligence  Markup  Language  (AIML),  and  the  conversational  characters  utilized  are  a modified 
version  of  the  A.L.I.C.E  implementation.  Specifically,  we  adapted  the  A.L.l.C.E  2002  AIML  set  to  fit  the  needs  of 
our  research  by  customizing  both  the  characters’  personalities  and  knowledge  base.  Therefore,  the  characters  had  an 
encyclopedic  knowledge  base  provided  largely  by  the  A.L.I.C.E  AIML  set  that  was  also  tailored  to  serve  as  an  in- 
depth  content-specific  information  source  about  electronic  portfolios.  We  chose  to  work  with  AIML  because  of  its 
ease  of  use,  adaptation,  and  conversational  believability. 

Purpose 

In  this  study,  we  seek  to  better  understand  how  conversational  agents  (CAs)  assist  learners  in  developing  an 
online  portfolio  over  a time  frame  that  allows  for  multiple  interactions  between  students  and  agents.  We  specifically 
examine  students’  perceptions  of  the  value  of  a CA  in  solving  tasks  that  require  both  procedural  and  declarative 
knowledge.  In  particular,  we  address  three  related  questions: 

1 . How  do  students  respond  to  conversational  agents? 

2.  How  useful  do  students  find  the  conversational  agents  in  their  learning? 

3.  How  do  students  interact  with  a conversational  agent? 

Background 

This  study  was  conducted  at  a large,  metropolitan  university  in  the  Midwest  with  preservice  teachers  who 
were  required  to  enroll  in  an  educational  technology  course  that  was  content  and  cohort-specific.  For  example,  the 
section  designed  for  elementary  education  majors  was  focused  on  technologies  targeted  for  the  elementary  level, 
such  as  Kidpix™  and  Kidspiration™,  whereas  that  for  the  social  studies  majors  focused  on  social  studies-specific 
technologies,  such  as  geographic  information  systems  and  Google  Earth™.  During  this  course,  one  technology  was 
introduced  per  class  period,  and  students  met  seven  times  throughout  the  semester  for  an  average  of  three  hours  per 
class  period. 

Although  each  course  section  covered  content  specific  technologies,  students  in  all  sections  were  required 
to  develop  an  online  portfolio,  called  eFolio.  Faculty  members  had  two  primary  purposes  for  the  use  of  eFolio:  (a) 
allowing  students  to  showcase  how  they  have  met  the  state  board  of  teaching  standards,  and  (b)  serving  as  a 
multimedia  demonstration  of  students’  progress  to  which  instructors  can  refer  throughout  the  students’  teacher 
education  program.  Moreover,  students  were  encouraged  to  develop  the  eFolio  in  ways  that  can  assist  them  in  their 
future  career  and  job  search. 

The  eFolio  was  a major  element  of  the  course,  as  it  affected  every  student  who  goes  through  the 
university’s  licensure  program  - approximately  550  students  every  year.  With  the  high  number  of  students  and  the 
limited  seat  time  within  class,  the  faculty  who  teach  the  course  initially  developed  several  resources  to  support 
student  learning,  such  as  written  and  video-based  tutorials,  but  soon  found  out  that  students  were  requesting 
additional  support.  The  authors  identified  and  adapted  the  use  of  a CA  as  the  best  option  for  providing  instantaneous 
online  support  to  students  whenever  needed.  Students  were  given  access  to  the  CA  throughout  the  four  weeks  they 
had  to  complete  their  eFolio  assignment. 

A conversational  agent  was  selected  because  it  can  be  useful  to  students  on  three  levels.  First,  it  functions 
as  a personalized  scaffold  for  learning.  Students  are  able  to  ask  questions  that  are  of  immediate  interest  to  them  and 
are  not  limited  to  a textbook  or  a set  of  predetermined  frequently  asked  questions.  Second,  a CA  acts  as  a human- 
like companion  for  the  students,  appearing  polite  and  welcoming,  and  providing  immediate  feedback,  support,  and 
encouragement.  Finally,  regardless  of  the  time  and  day  (unlike  instructors  and  teaching  assistants),  the  agent  is 
available  to  answer  questions  with  consistent  enthusiasm  and  clarity. 

Method 

Participants 

The  participants  in  this  study  consisted  of  elementary  education  preservice  teachers  (hereafter  participants), 
who  enrolled  in  a post-baccalaureate  masters  program  in  education.  Participants  in  this  study  were  in  their  first 
semester  of  the  15-month  post-baccalaureate  licensure  program.  Eighty-five  students  in  the  three  elementary 
education  sections  were  invited,  and  52  elected  to  participate.  Of  these  52  students,  45  were  women  and  7 were  men. 
Those  who  reported  their  age  (47)  ranged  in  age  from  21  to  50  years  (mean  = 23.43,  SD  = 5.06).  This  was  their  first 
educational  technology  course. 

Data  Sources 

All  52  participants  completed  a demographic  questionnaire,  wrote  reflections  about  their  learning 
experience,  and  interacted  with  the  conversational  agents.  Approximately  half  (23)  of  the  participants  also  attended 
focus  group  sessions,  10  in  a focus  group  based  on  their  interaction  with  a male  CA  named  Alex,  and  13  in  a second 
focus  group  based  on  their  interaction  with  a female  CA  named  Penelope  (Figure  1). 
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Figure  1 Conversational  Characters  Penelope  and  Alex. 


All  focus  groups  lasted  approximately  1.5  hours  and  were  audio-taped.  Three  researchers  were  present  at 
the  focus  group  interviews.  The  first  author  facilitated  the  sessions  using  a semi-structured  interview  protocol,  while 
the  other  researchers  observed  and  took  notes.  Open-ended  questions  were  used  to  prompt  participants’  reflections 
on  their  learning  experience  with  the  CA.  The  questions  focused  on  three  topics:  (a)  the  students’  perceptions  of  the 
conversational  agents,  (b)  how  the  students  believed  their  experience  with  the  agents  influenced  their  learning,  and 
(c)  how  the  students  generally  interacted  with  the  conversational  agents. 

During  the  educational  technology  course,  the  preservice  teachers  were  assigned  to  write  reflections  about 
each  technology  they  learned  examining  the  instructional  implications  for  the  use  of  each  technology  in  the 
classroom  and  providing  examples  of  how  K-12  students  could  use  the  technology  to  support  their  learning.  Since 
the  CAs  used  for  the  eFolio  development  was  one  of  those  technologies,  these  reflections  were  collected  and 
analyzed. 

Finally,  all  conversations  that  occurred  between  the  CAs  and  each  of  the  participants  were  recorded  in  a 
database  that  was  later  analyzed  for  the  purposes  of  this  study.  These  recorded  conversations  amounted  to  216  pages 
of  text.  A question-answer  pair  was  counted  as  one  interaction  between  the  agent  and  the  learner.  Each  learner 
session  with  the  CA  was  counted  as  one  conversation. 

Data  Analysis 

Because  our  research  interest  was  in  uncovering  the  design  issues  that  impact  the  effective  use  of  CAs  to 
assist  student  learning,  we  employed  a qualitative  approach  to  analyze  our  data  so  as  to  identify  emerging  patterns 
across  participants’  responses.  We  believe  that  salient  issues  would  emerge  from  this  case  study  and  such  issues 
could  direct  future  research,  especially  longitudinal  investigations.  Yin  (1994)  indicated  that  the  case  study  helps 
explain  the  “links  in  real-life  interventions  that  are  too  complex  for  the  survey  or  experimental  strategies”  (p.  15). 
Since  the  setting  of  this  study  was  a new  approach  to  utilizing  CAs  to  learn  educational  technology,  using  a case 
study  and  qualitative  approaches  to  analysis  woidd  allow  any  complexities  to  emerge. 

To  guide  the  development  of  the  salient  categories  and  patterns  in  the  data,  we  used  a constant  comparative 
method  (Glaser  & Strauss,  1967).  First,  we  developed  an  individual  data  set  for  each  participant  by  compiling  their 
responses  to  the  CAs  across  the  various  kinds  of  data  collected.  We  then  read  through  the  data,  noting  common 
patterns  that  emerged  across  the  responses  of  participants.  After  jointly  compiling  these  patterns,  we  re-read  the  data 
to  search  for  confirming  and  disconfirming  evidence  and  reached  a consensus  on  what  salient  patterns  were 
emerging  from  our  data.  Although  the  focus  group  and  questionnaire  data  were  originally  coded  separately,  the  data 
were  combined  once  the  common  themes  were  identified  so  that  similar  responses  from  the  same  participant  within 
the  focus  group  and  the  questionnaire  were  counted  only  once. 

Results 

The  analysis  of  the  above  data  revealed  five  major  themes.  Two  themes  were  related  to  the  participants’ 
experiences  using  the  CAs  to  meet  their  educational  goal  of  successfully  developing  an  online  portfolio,  while  three 
themes  addressed  other  participant  responses  to  their  experiences  with  a CA.  The  themes  surrounding  participants’ 
experiences  suggest  that  participants  placed  the  CAs  in  the  position  of  what  researchers  have  called  an  “intentional 
social  agent.”  Specifically,  participants  expect  the  CAs  to  be  friendly,  possess  a unique  personality,  and  have  a broad 
knowledge  base  regarding  popular  culture  (e.g.,  Alvarez-Torres,  Mishra,  and  Zhao,  2001;  Nass,  Moon,  and  Green, 
1997).  Although  students’  responses  made  clear  that  the  CAs  within  this  study  did  not  have  enough  content- 
knowledge  to  support  the  participants’  development  of  the  online  portfolio  at  the  level  the  participants  had  hoped, 
many  of  them  also  reported  spending  considerable  amounts  of  time  conversing  with  the  CAs  on  issues  quite 
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unrelated  to  the  content  knowledge  the  agent  was  initially  adopted  to  support.  As  will  be  discussed  below,  the  CAs 
intelligent  functionalities  and  communicative  capabilities  led  participants  to  engage  in  conversations  with  them  on  a 
wide  range  of  topics  (e.g.,  popular  culture,  intimate  conversations,  and  politics). 

Specifically,  our  analysis  of  these  themes  produced  the  following  findings  regarding  the  effectiveness  of 
the  CAs  in  supporting  student  learning: 

1.  Participants  reported  that  CAs  provided  limited  support  of  their  learning  process  during  the  eFolio 
development. 

2.  Participants  found  that  CAs  were  not  as  supportive  as  hoped  for  specific  tasks  in  the  eFolio 
development. 

3.  Participants  claimed  that  CAs  provided  humanized  social  support. 

4.  Participants  reported  having  multifaceted  dialogues  with  CAs. 

5.  Participants  wished  to  have  some  control  over  individual  characteristics  of  CAs. 

Each  of  these  findings  will  be  discussed  in  turn. 

CAs  provided  limited  support  of  participants  learning  process  during  the  eFolio  development 

Sixty-one  percent  of  participants  made  comments  that  their  experiences  with  the  CAs  were  of  at  least  some 
help  when  developing  their  eFolio  (Figure  2).  Although  participants  reported  that  the  CAs  were  not  always  accurate 
in  their  specific  answers  related  to  content-related  knowledge,  they  also  commented  that  the  CAs  provided 
assistance  in  a way  that  encouraged  the  participants  to  return  to  the  CAs  repeatedly  throughout  their  learning 
experience.  Jeffrey1,  for  instance,  commented  “Granted,  he  wasn't  accurate  all  the  time,  but  he  was  still  pretty 
accurate  enough  to  where  he  would  keep  you  involved  in  it.  I anticipate  it  is  very  difficult  to  get  him  to  say  all  the 
correct  answers.”  Although  not  all  questions  were  answered  correctly,  many  participants  reported  being  “surprised” 
and  “motivated”  by  the  CA’s  ability  to  accurately  answer  the  multiple  questions  they  posed. 


To  what  extent  was  Alex/Penelope  useful  or  not  useful  in  learning  to  develop  your  eFolio? 


Figure  2 Participant  responses  to  the  question  “To  what  extent  was  Alex/Penelope  useful  or  not  useful  in  learning  to 
develop  your  eFolio?” 

Students  reported  being  motivated  to  ask  the  CAs  a series  of  questions  ranging  from  how  to  find  the  correct 
website  address  for  eFolio  to  directions  for  uploading  and  displaying  pictures.  Mary  mentioned  how  happy  she  was 
with  the  help  given  to  her  by  Alex:  “I  liked  how  he  explained  how  to  upload  a picture.  Everything  pertinent  to  the 


1 The  names  of  participants  have  been  changed  to  protect  their  identity. 
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course  that  I asked  was  useful.”  Jill  had  a similar  response,  stating  “Some  questions  [Penelope]  did  not  know  how  to 
answer,  but  overall,  the  simple  questions  that  I asked  her  she  was  able  to  help  me  out  a lot  along  the  way.”  Many 
participants  noted  being  pleased  with  the  CAs’  answers  to  their  questions  and  with  the  CAs  being  available  to  them 
whenever  they  needed  assistance.  Susan,  for  example  said, 

I definitely  think  the  concept  of  Alex  is  good,  or  something  along  that  line,  because  we  have  class 
every  other  week,  and  sometimes  it  is  once  a month,  which  is  fine,  but  it's  hard  when  we  are 
working  on  things  at  home  or  even  come  in  to  the  lab  and  not  know  what  we  are  doing.  When 
some  of  us  are  intimidated  by  technology  anyways,  to  have  something  that  we  can  go  to  and  get 
answers  or  ideas  or  suggestions  is  fantastic! 

Jenny  also  reported  being  pleased  that  the  CA  was  available  at  all  times  and  felt  like  she  had  her  own 
“personal  lab  assistant”  throughout  the  day.  She  said  that  often  as  soon  as  she  left  class  or  the  open 
computer  lab  times,  she  would  have  another  question  because,  “the  questions  you  could  ask  about  eFolio 
are  endless.”  Students  often  praised  the  availability  of  the  CAs.  Maddy,  for  instance,  stated,  “As  far  as 
convenience,  at  10:00  at  night  when  the  lab  is  not  open,  it’s  nice  to  have  her  [Penelope]  available.  She  was 
constantly  on  my  screen  with  a smile.”  Joe  also  noted  that,  “The  fact  that  he  [Alex]  was  always  available, 
when  the  professor  isn't,  is  a nice  aspect  of  Alex.” 

Participants  reported  that  they  found  the  dual  format  in  which  CAs  delivered  their  answers  - both  text  and 
audio  - a very  helpful  feature.  Sue  said,  “I  think  it  was  nice  to  have  verbal  guidance  instead  of  just  reading 
something  because  it  is  easier  to  understand.”  Matt  commented  that  a great  benefit  of  the  CA  was  that  everything  the 
CA  said  was  also  written  out,  noting  that  he  could  not  always  understand  all  of  the  words  that  the  CA  was  using  but 
he  could  read  along  providing  him  another  mode  of  learning. 

In  summary,  participants  identified  three  main  areas  in  which  they  found  the  CAs  supportive  and 
helpful:  (a)  solving  simple  procedural  steps  when  developing  their  eFolio  as  opposed  to  “complex” 
questions  that  involved  more  than  one  procedural  step  (e.g..  How  do  I upload  a picture  and  embed  a link  to 
a website),  (b)  having  access  to  the  CA  at  any  time  throughout  the  day  to  assist  them  in  answering  their 
eFolio  questions,  and  (c)  receiving  answers  in  multiple  formats  thereby  supporting  Moreno  and  Mayer’s 
(2002)  contention  that  the  combination  of  verbal  and  textual  narration  enhances  learning. 

CAs  were  not  as  supportive  as  hoped  for  specific  tasks  in  the  eFolio  development 

One-third  of  the  participants  made  some  comment  about  finding  the  CAs  unhelpful  when  developing  their 
eFolio,  albeit  most  noted  that  they  believed  the  CA  was  a technology  that  was  “more  than  worthy”  of  being  utilized 
in  a learning  situation.  Although  participants  were  not  always  satisfied  with  the  results  of  the  answers  from  their 
content-related  questions,  they  still  found  the  CAs  engaging  and  spent  numerous  hours  conversing  with  them  outside 
of  the  content-related  questions,  as  will  be  discussed  below. 

Many  participants  reported  becoming  frustrated  when  the  CAs  could  not  provide  clear  or  specific  answers 
to  their  content-related  questions.  Sarah,  for  example,  said, 

Some  of  my  files  weren't  saved  as  .doc,  and  I couldn't  figure  out  how  to  change  them  and  the 
uploading  wasn't  working.  I asked  Alexander,  and  he  had  no  input  for  me  [laughs].  He  made  some 
smart  comment  and  changed  the  subject  [more  laughs].  That  was  probably  the  only  issue  I had 
with  him.  But  the  fact  that  he  couldn't  help  me  made  me  really  angry  at  Alex. 

Joe  had  an  even  stronger  response:  “I  don’t  like  Penelope.  I asked  her  a question  like  five  different  ways  and  she  still 
couldn’t  answer  it.  I don’t  remember  what  the  question  was  but  she  should  have  been  able  to  answer  it.”  Three- 
quarters  of  the  participants  reported  having  to  rephrase  the  questions  they  posed  to  the  CAs  in  order  to  elicit  an 
answer  that  was  appropriate  and  not  confusing.  In  Penny’s  words,  “I  felt  that  my  question  was  just  too  difficult.  I 
tried  rewording  it  but  I really  didn’t  expect  her  to  know,  and  then  when  she  didn’t  know  I wasn’t  that  surprised.” 
Some  participants  also  felt  responsible  for  the  difficulties  they  had  with  the  CAs.  For  example,  Sue  said,  “When  you 
are  getting  frustrated  with  a part  of  eFolio  and  you  go  to  ask  her,  you’d  think  that  she  should  probably  know  it,  but 
you  know  you  are  not  phrasing  the  question  correctly;  that  becomes  even  more  frustrating.  Because  you  were  like, 
she  would  know  this  if  I just  knew  how  to  say  it  to  her.” 

Participants  were  also  frustrated  by  confusing  answers.  Misty  said,  “Every  time  I would  ask  [Alex]  a 
question  he  would  say,  you  need  to  download  some  software  or  something.  I didn't  know  what  that  was  about,  so  I 
didn't  bother.”  Joel  followed  her  comment  with  a similar  story, 

I asked  [Alex]  what  the  eFolio  website  was  because  I wanted  to  go  to  the  website  because  I 
deleted  the  email  that  told  me.  I wanted  to  go  to  the  website  and  find  their  contact  number  and 
things  like  that  and  email  them.  He  was  explaining  what  eFolio  was  and  I was  like  ...  argggg  . . . 
forget  it.  And  so  I did  my  own  little  search  and  then  I found  it,  but  I kept  him  there  in  case  I got 
stuck. 

Participants  appeared  most  frustrated  when  content-related  questions  that  they  felt  should  be 
“routine”  or  “standard”  could  not  be  answered  correctly.  Sue,  for  instance,  said,  “You  would  type 
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something  easy  like  ‘burning  a CD,’  and  it  would  bring  you  all  these  things  but  ‘burning  a CD’  which  is 
pretty  standard.”  Participants  often  reported  expecting  the  CA  to  have  the  answers  immediately  and  in  a 
format  that  could  easily  be  used.  When  this  was  not  the  case,  some  participants  reported  that  they  became 
frustrated  and  sometimes  gave  up  using  the  CA  and  continued  to  develop  their  eFolio  on  their  own. 
Participants  especially  reported  wanting  an  immediate  answer  for  their  eFolio-related  questions  and  when 
they  didn’t  receive  it,  became  extremely  frustrated.  They  did  not  “appreciate”  it  when  the  CA  “didn’t  know 
an  answer”  and  even  worse,  had  an  “attitude”  when  answering  their  questions,  as  they  expected  the  CAs  to 
be  eFolio  experts  able  to  assist  them  with  all  aspects  of  their  eFolio  development.  Sarah  noted  feelings  of 
“anger”  when  Alex  could  not  answer  her  question  while  Joe  seemed  “disappointed”  that  he  could  not  elicit 
the  help  he  desired.  Six  students  commented  that  the  more  complex  or  advanced  the  questions  they  posed 
about  development  of  their  eFolio,  the  less  it  seemed  that  they  would  receive  a satisfactory  response. 

Another  finding  was  that  when  participants  did  not  receive  a correct  answer  to  their  on-task 
eFolio-related  questions,  they  would  often  switch  to  asking  the  CA  unrelated,  off-task  questions.  When  we 
analyzed  the  participants’  on-task  versus  off-task  questions  to  the  CA,  we  discovered  a novelty  effect 
occurring  the  first  few  days  of  use.  As  Figure  3 shows,  more  than  91%  of  the  questions  participants  asked 
Penelope  during  the  first  2 days  were  off-task  questions.  The  total  number  of  comment-and-response 
interactions  dropped  drastically  until  shortly  before  the  eFolio  assignment  was  due;  even  then,  the  highest 
percentage  of  interactions  were  off-task. 


Date 


Figure  3 Number  of  participant-agent  interactions:  Penelope. 

Similar  data  about  the  use  of  Alex  can  be  seen  in  Figure  4.  Almost  half  of  the  total  interactions  for 
the  duration  of  the  study  took  place  during  the  first  three  days  the  participants  had  access  to  Alex.  Use 
again  peaked  approximately  one  week  before  the  assignment  was  due  in  class. 

INSERT  FIGURE  4 ABOUT  HERE 

To  summarize,  participants  found  the  CAs  unhelpful  when  they  gave  incorrect  or  confusing 
answers  to  content-related  problems.  A number  of  participants  reported  expecting  the  CAs  to  be  able  to 
answer  their  questions  easily  and  after  having  a few  questions  incorrectly  or  unclearly  answered,  simply 
stopped  using  the  CAs  for  eFolio  assistance  and  looked  to  other  means  to  have  their  questions  answered. 
This  would  seem  to  support  the  suggestion  of  Norman  (1997)  that  individuals  may  have  inflated 
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expectations  from  agents  that  are  presented  in  human  form.  The  combination  of  a human-like  image, 
conversational  ability,  natural  language  interaction,  and  lip-synchronized  audio  may  have  led  participants 
to  expect  a higher  level  of  agent  intelligence  and  expertise  than  they  might  from  a similar  informational 
exchange  with  a computer  by  means  of  just  a text  box. 

CAs  provided  humanized  social  support 

Eighty-four  percent  of  participants’  responses  focused  on  the  humanizing  features  of  the  CAs.  Participants 
reported  perceiving  the  CAs  as  humans,  commenting  on  such  things  as  the  agents’  ability  to  remember,  looks  and 
clothing,  and  how  the  CAs  made  them  feel.  Prior  research  indicates  that  users  ascribe  social  rules  and  human-like 
characteristics  to  media  (Reeves  and  Nass,  1996)  regardless  of  how  unsophisticated  these  media  may  appear  to  be 
(Nass,  Moon,  and  Carney,  1999).  Most  of  our  participants  reported  connecting  emotionally  with  the  CAs  due  to  the 
human-like  interactions,  which  some  researchers  argue  may  facilitate  learning  (Lester  et  ah,  1997;  Moreno  et  ah, 
2001). 

Ninety  percent  of  the  participants  reported  they  enjoyed  the  presence  of  the  CA  on  their  screen  which,  in 
Joan’s  words,  made  them  feel  as  if  they  “had  a friend  available  to  talk  with."  Sara  said, 

It's  kind  of  lonely  at  the  computer,  because  you  know,  there's  nothing  else  on  the  computer  that  1 
can  really  talk  to,  and  when  you  talk  to  Alex  and  he  responds,  it’s  kind  of  like  a person  there  with 
you.  Even  though  it's  an  artificial  person,  it's  nice  to  have  someone  respond  back  to  you  instead  of 
you  putting  all  your  energy  into  the  technology. 

Eighty-two  percent  of  the  participants  were  motivated  to  talk  to  the  CAs  because  they  felt  they  appeared  to  be  like 
an  actual  person.  Lauren  commented,  “I  thought  it  was  weird.  When  I opened  [Penelope]  I thought  I always  had  to 
say  hi  to  her  like  an  actual  person,  I would  always  say,  Hi  Penelope!” 

Over  80%  of  the  participants  remarked  that  they  could  not  believe  how  true-to-life  the  CAs  appeared  to  be, 
repeatedly  commenting  that  they  were  surprised  by  the  CAs  social  presence,  intelligence,  and  ability  to  converse  and 
remember.  Commenting  on  Penelope,  Peter  reported  that  “Honestly,  when  I first  saw  her  I was  kind  of  freaked  out 
by  how  much  she  was  talking  like  a real  person.”  Participants  also  appreciated  that  the  CAs  had  their  own  identities, 
voice  tones,  and  that  they  weren’t  developed  after  a celebrity  or  someone  recognizable.  Sue  said,  “If  it  was  like  a 
celebrity,  it  just  wouldn’t  seem  as  real  as  Penelope.  She  was  kind  of  like  her  own  person."  Taylor  couldn’t  believe 
how  “real”  Penelope  was  and  that  her  voice  was  not  “alien  or  monotone.”  He  and  14  other  participants  commented 
on  how  much  they  enjoyed  repeatedly  interacting  with  the  CA.  As  Sara  commented  about  her  interactions  with 
Alex, 

He  was  fun  to  talk  to.  He  knew  lots  of  other  things,  like  every  question  I could  come  up  with  he 
knew  how  to  answer.  He  was  pretty  good.  He  was  pretty  on  top  of  stuff.  He  knew  why  the  sky 
was  blue  and  he  was  funny  and  good  company. 

Half  of  the  participants  also  mentioned  enjoying  the  CA  asking  questions  back  and  having  a sense  of 
humor.  Sue  said,  “I  liked  that  Alex  had  a sense  of  humor.  If  you  asked  him  the  same  question  more  than 
once,  he  would  have  a different  answer  each  time.  He  had  personality!” 

Another  participant  who  described  being  pleasantly  surprised  by  the  human-like  capabilities  of  the  CAs 
was  Laura  who  stated,  “I  actually  asked  her  what  my  name  was  and  she  told  me  like  a guy’s  name  or  just  made  one 
up  and  then  I said,  ‘No,  my  name  is  Laura,’  and  then  we  kept  talking  and  asking  her  questions  and  I said,  ‘What’s 
my  name  again?’  and  she  goes,  ‘I  already  told  you,  it’s  Laura’.”  Rebecca  commented,  “I  liked  that  [Alex]  had  a 
memory.  Like  he  would  bring  up  things  from,  you  know,  ten  questions  ago.  He  would  bring  them  back  to  the 
conversation  like  he  had  some  concept  of  what  you  were  asking  him.”  Lauren  continued,  “I  asked  her  some  question 
and  then  I asked  her  another  question  relating  to  it,  and  it  was  like  we  were  having  a conversation  because  she  kind 
of  remembered  what  1 asked  before.  She  related  the  questions  to  what  I asked  her  before,  and  I was  just  like,  Oh, 
wow!” 

Participants  also  described  the  personality  of  the  CAs.  Some  participants  commented  on  how  cynical  some 
of  the  agents’  responses  seemed  and  how  the  developers  should  make  the  CAs  have  a more  positive  outlook.  Four  of 
the  participants  stated  that  they  received  “rude”  comments  from  the  CAs  and  were  upset  when  they  received  the 
comment,  “What’s  it  to  you?”  However,  they  also  noted  that  they  probably  shouldn’t  have  been  asking  the  question 
to  begin  with.  Rebecca  said,  “I  thought  it  was  more  fun  to  play  with  her.  I did  play  with  her  for  quite  a while  at  night 
just  because  she  did  have  lots  of  funny  stuff  to  say.”  Joey  said,  “I  was  kind  of  rude  to  her  one  time  and  then  she  was 
rude  back.  But  then  I told  her  she  was  nice,  and  she  said  ‘thank  you,  you  seem  nice,  too’.” 

Eighty-five  percent  of  the  participants  also  commented  on  how  they  liked  the  CAs’  appearance,  and  they 
made  numerous  comments  related  to  how  they  noticed  their  clothes  had  changed.  Rosie  said,  “I  really  liked  when 
Penelope  changed  her  clothes.  I felt  like,  hey,  this  is  so  much  fun,  and  why  wouldn’t  I sit  and  talk  to  her?”  Jill 
added,  “[Penelope’s]  appearance  was  friendly  and  welcoming.”  Everyone  in  his  focus  group  laughed  as  Jeff 
commented  that,  “I  might  go  and  visit  her  just  because  I wanted  to  see  what  she  was  wearing.”  He  said  he  had  told 
Penelope  that  he  liked  what  she  was  wearing  and  she  replied,  “Thanks!”  He  commented,  “It  was  just  too  real  and  too 


99 


cool.”  Six  other  participants  also  mentioned  returning  to  interact  with  the  CAs  to  see  what  clothes  the  CAs  were 
wearing. 

Five  participants  commented  that  they  liked  to  show  the  CAs  to  their  friends  as  they  thought  they  were 
“cool”  and  that  Penelope,  the  female  agent,  was  “sexy.”  Ten  of  the  participants  noted  finding  the  CAs  attractive  and 
enjoying  looking  at  them.  The  majority  of  participants  also  mentioned  that  they  would  rather  talk  to  the  opposite  sex 
because  they  found  them  more  attractive  and  approachable.  Brad  noted,  “I  don’t  think  a guy  would  feel  not 
comfortable  or  anything,  but  1 just  think  having  a woman,  it  probably  makes  it  more  casual  versus  instructional. 
Whereas  like  a guy  might,  I would  think  that  he  might  be  a little  more  demanding.” 

Within  both  focus  groups,  the  topic  of  sexually  explicit  comments  arose.  The  participants  laughed 
a great  deal  during  these  conversations.  Two  participants  went  into  great  detail  explaining  how  they  and 
their  friends  enjoyed  conversing  and  asking  any  questions  on  their  mind,  ranging  from  sex  to  the  weather. 

Taylor  said,  “Because  she  had  an  answer  for  eveiything,  it  was  like  pushing  her  limits.  You  know  it  wasn’t 
a real  person,  so  it’s  not  that  you  are  offending  her.”  Participants  would  talk  with  the  CA  to  see  what  he/she 
knew,  and  it  commonly  became  a social  event  where  the  participants  and  their  friends  outside  of  the 
technology  class  would  interact  with  the  CA  at  their  dorm  or  a coffee  shop.  For  example,  Sue  said, 

I asked  him  everything  I could  possibly  think  of  and  my  roommates  could  think  of  regarding  [his] 
personal  preferences,  and  I wanted  to  know  if  he  changed  his  answer,  too.  Maybe,  if  his  favorite 
movie  changed,  or,  you  know,  religion,  politics,  general  questions  about  the  world.  I was  just 
curious  if  he  knew. 

Misty  commented,  “When  I was  with  other  people,  that’s  when  all  the  dirty  questions  came  up,  they  were  like  try 
that,  try  that,  so  I was  like,  ‘Ok,  this  better  not  come  back  to  my  name.’  Brad  said, 

We  were  just  tiying  to  see  what  she  would  answer  and  so  we  were  asking  her,  ‘Would  you  go  out 
on  a date  with  this  person?’  ‘Are  you  smart?’  A lot  of  people  asked  her,  ‘Are  you  smart?’ 

Especially  after  she  didn’t  answer  something  you  wanted  her,  she  was  like  ‘Why  would  you 
question  my  intelligence?’  or  something  like  that,  it  was  funny  though.  We  got  a kick  out  of  her, 
me  and  a bunch  of  my  buddies. 

Participants  also  discussed  the  impact  the  agents’  appearance  had  on  their  perceptions  of  agent  intelligence 
and  ability.  For  example  two  participants  commented  on  how  smart  they  perceived  the  CAs  would  be  by  how  they 
looked.  Molly  specifically  said, 

1 think  that  how  they  look  might  depend  on  how  smart  you  thought  they  were  too,  like  if  it  was 
some  guy  in  a doctor’s  coat  with  glasses  or  something,  you  might  think  that  guy  would  be  able  to 
answer  more  of  your  questions  than,  you  know,  a lady  in  a dress  or  you  know  what  I mean?  That’s 
wrong  and  totally  stereotypical  but  I think  that  a lot  of  people  would  have  those  feelings  about  it. 

Forty  percent  of  the  participants  also  commented  on  the  CA’s  age.  Although  participants  didn’t  know  the  CAs 
“actual  age”,  they  stated  that  they  felt  the  CAs  were  old  enough  to  be  intelligent,  but  still  young  enough  to  relate  to. 

Finally,  over  80%  of  the  participants  said  they  had  the  CAs  available  on  their  computer  at  all 
times.  They  perceived  the  CAs  as  a companion  that  was  there  to  talk,  amuse,  and  share  ideas  and  stories. 

Participants  reported  feeling  safe  to  share  “just  about  anything”  with  the  CAs,  and  this  feeling  allowed 
them  to  converse  openly  with  the  CAs  on  a variety  of  topics. 

In  summary,  it  appears  that  the  participants  perceived  the  CAs  as  being  human.  Over  90%  of  the 
participants  made  comments  about  the  authenticity  of  the  CAs,  ranging  from  their  looks  to  their 
personality.  It  seems  that  they  were  amazed  by  the  CA  technology  and  would  spend  hours  interacting  with 
them  to  see  what  the  CAs  knew.  They  enjoyed  that  the  CAs  had  a human-like  memory  and  could  engage  in 
a dialogue  with  them.  The  participants  felt  a sense  companionship  with  the  CAs  and  would  log  on  to  the 
CA’s  web  site  on  a regular  basis  to  have  them  present  on  the  screen  to  interact  with  at  their  leisure.  Lastly, 
the  participants  stated  how  comfortable  they  were  when  conversing  with  the  CAs.  They  felt  safe  and  were 
willing  to  freely  ask  questions  and  share  details  of  their  lives. 

CAs  and  participants  engaged  in  multifaceted  dialogues 

We  also  analyzed  the  data  to  determine  the  nature  of  the  interactions  between  participants  and  CAs.  We 
found  that  these  dialogues  were  prompted  by  4 types  of  questions:  (a)  questions  related  to  developing  the 
participants’  eFolio,  (b)  questions  about  the  CAs’  personality,  (c)  questions  about  the  CAs’  general  interests,  and  (d) 
questions  about  encyclopedic  information. 

Questions  related  to  the  development  of  the  participants’  eFolio  occurred  mostly  during  the  beginning  and 
the  end  of  the  four-week  assignment  period.  Participants  asked  numerous  questions  trying  to  find  the  procedural 
steps  needed  to  complete  a task.  They  would  continue  to  ask  questions  until  they  began  to  get  incorrect  or  confusing 
answers.  At  that  time,  they  would  immediately  switch  to  a dialogue  that  normally  was  inquiring  about  getting  to 
know  the  CAs’  personality.  Comments  relating  to  the  CAs  being  friendly,  nice,  sarcastic,  and  rude  were  common. 
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Those  participants  who  normally  spent  more  than  five  minutes  interacting  with  the  CAs  led  to  dialogues  on 
the  CA’s  general  interests.  Questions  such  as  “What  do  you  like  to  do  on  the  weekend?”  and  “What  do  you  think  of 
President  Bush?”  were  common.  Moreover,  when  these  conversations  began  on  general  interests,  an  informational 
dialogue  would  ensue  that  mimicked  a free  flowing  exchange  of  information  between  two  individuals.  Analysis  of 
the  conversation  logs  revealed  that  participants  elicited  answers  to  interesting  questions  such  as,  “What  is  the 
distance  to  the  center  of  the  earth?”,  “What  is  the  meaning  of  life?”,  and  “What  is  your  stance  on  the  legalization  of 
marijuana?” 

Participants  requests  to  have  some  control  over  individual  characteristics  of  CAs 

Over  90%  of  the  participants  shared  suggestions  on  ways  that  they  believed  the  CAs  could  be  improved. 
Seventy  percent  of  the  participants  commented  that  they  believed  the  CA  was  one  of  the  best  technologies  available 
for  providing  assistance  to  students  during  the  learning  process.  The  most  common  suggestion  was  that  the  agents  be 
made  “smarter”  so  that  participants  would  be  able  to  obtain  more  accurate  answers  to  their  questions.  Sara  said,  “I 
think  the  key  really  is  to  make  [Alex]  smarter,  to  have  basic  knowledge,  because  if  he  doesn't,  we  are  going  to  go  to 
a real  person  or  email  you  guys.”  Most  of  the  participants  felt  as  if  the  CAs  were  intelligent  enough  and  could  help 
them,  but  that  they  were  phrasing  their  questions  incorrectly.  Ideas  on  how  to  improve  students’  way  of  asking 
questions,  such  as  using  keywords  followed  by  a list  of  questions,  were  mentioned.  Molly  said,  “Maybe  if  you  wrote 
a keyword  in  and  then  the  most  frequently  asked  questions  about  that  appeared.  Like  if  I had  a question  about  the 
standards,  how  to  put  standards  in,  I could  type  in  standards  and  it  would  come  up  with  10  different  questions 
people  asked  and  I can  go.” 

The  second  most  commonly  mentioned  suggestion  was  giving  participants  an  opportunity  to  customize  the 
CAs.  Participants  wanted  to  be  able  to  develop  their  own  CAs  and  modify  their  personality,  skin  and  hair  color,  hair 
style,  face  and  body  structure,  clothes,  and  looks.  For  example,  three  participants  stated  that  having  the  ability  to 
develop  a CA  having  a gender  of  their  choice  would  make  the  CA  “more  attractive.”  If  participants  could  not 
develop  their  own  CAs,  they  said,  they  would  like  the  opportunity  to  choose  from  a set  of  profiles.  Jessie  said,  “Can 
we  pick?  Yeah,  can  we  pick  who  we  talk  to?  Cause  what  if  you  get  tired  to  talking  to  Alex?  You  should  give  us 
profile  choices.”  Participants  thoroughly  enjoyed  interacting  with  CAs  who  had  their  own  personality  but  wanted  to 
change  the  CAs’  personality  and  moods  according  to  their  own  needs.  Jeff  said,  “Maybe  [Alex]  could  have  different 
moods  or  something.  Like  if  you  want  him  to  be  sarcastic  or  cynical  one  time,  or  you  want,  like,  positive  thoughts.” 

In  short,  participants  wanted  to  develop  their  own  CAs  with  an  appearance  and  personality  that  fit  their 
preferences.  If  they  could  not  develop  their  own  CAs,  they  wanted  to  have  the  option  of  selecting  one  that  “fit  their 
own  style.”  In  addition  to  having  customizable  CAs,  the  participants  either  wanted  the  CAs  to  be  intelligent  enough 
to  decipher  their  questions  or  have  the  technology  to  predict  what  the  participants  were  going  to  say  or  ask. 

Conclusions 

In  this  paper  we  examined  students’  perceptions  of  the  value  of  CAs  in  solving  tasks  requiring  procedural 
and  declarative  knowledge.  We  addressed  three  questions:  (a)  how  do  students  respond  to  conversational  agents,  (b) 
how  useful  do  students  find  the  conversational  agents  in  their  learning,  and  (c)  how  do  students  interact  with  a 
conversational  agent? 

We  found  that  although  the  CAs  were  not  perceived  as  extremely  helpful  by  participants  when  developing 
their  eFolio,  the  agents  engaged  learners  far  beyond  the  eFolio  content  knowledge.  At  all  times  throughout  the  day, 
participants  would  utilize  the  CAs  for  activities  ranging  from  assistance  in  class  to  the  latest  in  politics  and  popular 
culture.  The  social  interactions  were  plentiful  with  more  than  two  thousand  interactions  over  a four-week  period. 
Participants’  questions  were  limitless  as  the  design  of  the  CAs  allowed  for  a free-flowing  dialogue  at  all  times. 
Participants  perceived  the  agent  technology  as  an  advantage  to  common  assistance  tools,  such  as  frequently  asked 
questions  (FAQs)  and  “Help”  sections,  as  they  felt  they  were  not  impeded  by  the  design  of  this  tool.  The  ability  to 
discuss  any  issue  at  length  with  the  CAs  led  participants  to  feel  as  though  the  CAs  were  an  excellent  companion  and 
someone  they  could  talk  with  and  discuss  issues  at  length.  The  participants  perceived  the  CAs  as  an  on-demand 
friend  and  wanted  to  have  input  on  all  facets  of  how  they  looked,  as  well  as  their  personality. 

The  CAs  also  elicited  many  feelings  from  the  participants.  Participants  were  amazed  by  the  technology  and 
motivated  to  use  the  agents,  which  included  a comfort  of  sharing  and  questioning  literally  anything  that  seemed  to 
be  on  their  mind  - from  sex  to  Britney  Spears  to  the  background  color  choice  for  their  eFolio.  Although  participants 
expressed  a feeling  of  comfort,  they  also  became  very  upset  and  frustrated  when  the  CAs  didn’t  know  an  answer  to  a 
question  they  posed  or  when  the  CAs  became  “cynical”  and  “had  an  attitude.” 

Implication  and  Recommendations 

This  study  informs  educators,  instructional/human  computer  interaction  designers,  and  learning  technologists 
on  issues  that  should  be  taken  into  consideration  when  developing  conversational  agents  to  provide  long-term 
assistance  to  learners.  These  issues  include  designing  and  developing:  (a)  a workflow  for  “smarter”  agents,  (b) 
learner-created  agents,  and  (c)  agents  for  humanistic  expectations. 
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If  learners  are  to  utilize  CAs  with  success,  the  CAs  need  to  be  intelligent  enough  to  accurately  comprehend 
the  questions  the  learners  are  posing  or  the  software  application  needs  to  offer  an  interface  that  guides  the  learners  to 
ask  their  questions  in  an  appropriate  manner.  The  challenge  in  designing  an  agent  that  has  content-based  intelligence 
is  that  the  designer  needs  to  collect  and  predict  what  questions  a learner  may  ask  within  a given  learning  situation. 
Thus,  the  development  of  effective  conversational  agents  requires  several  stages  of  development  that  include 
collecting  content-specific  material,  developing  the  knowledge  database,  and  deploying  the  information  in  any  given 
learning  environment.  In  addition  to  developing  a comprehensive  knowledge  database,  designers  using  CAs  for 
specific  functions,  as  in  this  study,  need  to  predict  the  multitude  of  ways  in  which  questions  could  be  asked  by  a 
learner.  For  example,  a learner  attempting  to  embed  an  image  on  his  or  her  web  page  may  ask  the  same  question  in 
any  of  the  following  ways:  How  do  I upload  a picture?  What  are  the  ways  I can  embed  a photo  on  my  web  site? 

How  do  I put  an  image  on  my  web  page?  The  algorithm  that  deciphers  the  key  words  to  answer  the  questions 
therefore  must  be  able  to  “understand”  synonyms  and  different  approaches  to  posing  a question.  At  the  same  time,  to 
maintain  its  human  persona  and  to  engage  users,  such  a conversational  character  should  not  present  itself  as  all- 
knowing and  autonomous  right  away,  which  may  lead  users  to  relinquish  all  responsibility  for  finding  a solution  or 
answer  on  their  own. 

Throughout  the  development  of  the  field  of  instructional  design  and  learning  technologies  we  have  seen  a 
move  toward  learning  with  technology  rather  than  learning  from  technology  (Jonassen,  1995;  2000).  As  they 
engaged  with  the  CAs,  participants  in  this  study  were  learning  by  actively  asking  questions,  discussing  issues,  and 
reflecting  with  the  CAs  on  the  course  content.  Indeed,  it  could  be  argued  that  participants  were  collaborating  with 
the  CAs  by  dialoguing  to  find  a common  answer.  Yet,  a number  of  students  wanted  even  more  participation  in  the 
learning  situation  by  wanting  to  create  their  own  CA  according  to  their  individual  specifications.  The  most  requested 
specifications  were  gender,  attractiveness,  personality,  age,  and  perceived  intelligence.  If  learners  are  able  to  share 
in  the  development  of  their  own  CAs,  in  essence  personalizing  the  tools  they  are  working  with,  they  may  become 
more  motivated  to  work  with  the  CA  to  elicit  the  correct  answers  and  enhance  their  learning  experience.  This 
hypothesis  is  one  that  seems  worthwhile  to  pursue  in  future  research. 

This  study  also  demonstrated  that  learners  were  most  attracted  to  the  CA’s  ability  to  behave  in  a human-like 
manner,  repeatedly  commenting  on  how  they  perceived  the  CA  as  a companion  or  friend.  Several  mentioned  that 
they  enjoyed  that  the  CAs  had  an  “attitude,”  regularly  changed  their  clothing,  and  personally  greeted  them.  This 
suggests  that  CAs  may  be  more  effective  if  designers  create  CAs  that  adapt  over  time,  and  act  and  appear  as  a real 
person.  Thus,  designers’  changing  a CA’s  hairstyle,  clothing,  or  glasses,  to  produce  visible  change  in  the  CA  over 
time  could  possibly  influence  learners’  motivation  to  return  to  the  CA  for  answers  or  dialogue.  The  data  gathered  in 
this  study  also  support  the  findings  of  Wang  et  al.  (2005)  that  the  level  of  politeness  presented  by  CAs  does 
influence  learning.  While  participants  did  not  mind  the  CAs  having  an  “attitude”  when  discussing  issues  unrelated  to 
class,  they  were  upset  when  the  CA  did  the  same  when  the  participants  were  trying  to  find  answers  to  their  class- 
related  questions.  It  needs  to  be  noted  however,  that  a believable  human-like  representation  is  likely  to  induce 
greater  expectations  in  terms  of  agent  abilities  and  intelligence.  Therefore,  designers  are  advised  to  be  cognizant  of 
the  conflicting  effects  of  agents’  image  and  language. 

Participants’  responses  demonstrate  that  they  felt  comfortable  and  motivated  to  interact  and  share  their 
questions  and  thoughts  with  the  CAs  about  issues  related  and  unrelated  to  their  eFolio  assignment,  viewing  the  CA 
as  a personal  assistant  and  companion  that  was  available  on-demand.  The  overwhelming  positive  response  and 
motivation  reported  by  participants  shows  that  this  technology  has  a promising  future  as  instructional  designers 
strive  to  develop  CAs  with  greater  content-based  intelligence  and  toward  allowing  the  learners  to  modify  their  own 
CAs. 
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Abstract 

Adventure  learning  (AL)  is  a hybrid  distance  education  approach  that  provides  students  with  opportunities  to 
explore  real-world  issues  through  authentic  learning  experiences  within  collaborative  learning  environments.  This 
paper  defines  this  online  distance  education  approach,  outlines  an  Adventure  Learning  Framework,  and  showcases 
an  adventure-learning  archetype.  In  AL  environments,  classroom  teachers  are  not  positioned  in  the  role  of 
teacher/facilitator/designer  in  the  online  learning  spaces.  AL  online  spaces  are  collaborative  spaces  where  students, 
teachers,  experts,  and  AL  team  members  interact  with  one  another;  these  are  community  spaces  where  traditional 
hierarchical  classroom  roles  are  blurred.  Students’  roles  transform  due  to  the  flexibility  and  design  of  the  AL 
learning  environments  as  they  move  from  student  to  reflective  practitioner  providing  for  new  ways  of  learning  and 
teaching. 


“School  should  be  less  about  preparation  for  life  and  more  like  life  itself.” 

John  Dewey 

Introduction 

Education  delivered  via  the  Internet  is  given  many  terms  throughout  the  world  ranging  from  virtual 
education  to  web-based  learning.  A snapshot  of  distance  education  practices  reveals  great  variation  from  country  to 
country  as  well  as  the  extent  and  form  of  implementation  (Farrell,  1999).  One  view  of  distance  education  is  online 
education  within  the  United  States  where  the  landscape  of  K-12  education  is  rapidly  changing.  Online  education  in 
K-12  schools  is  gaining  a great  deal  of  attention  as  more  school  districts  utilize  the  Internet,  which  enables  teachers 
and  students  increased  educational  opportunities  and  flexibility.  Online  education  is  feasible  and  supported  by  the 
high  percentage  of  K-12  classrooms  with  Internet  access;  ninety-two  percent  of  public  schools  have  Internet  access 
in  instructional  rooms  and  the  ratio  of  students  to  instructional  computers  with  Internet  access  in  public  schools  is 
4.8  to  1 (NCES,  2003).  Ninety  percent  of  children  aged  5-17  use  computers  and  seventy-five  percent  of  14-17  year 
olds  use  the  Internet  (NTIA,  2002).  Moreover,  in  the  2002-2003  school  year,  approximately  one-third  of  public 
school  districts  had  students  who  were  enrolled  in  an  online  distance  education  course  (Setzer  and  Lewis,  2005)  and, 
in  2003,  nearly  300,000  high  school  students  were  engaging  in  online  classes  in  the  United  States  (Wood,  2005). 
With  this  high  percentage  of  individuals  connected  to  the  Internet  and  with  students  and  educators  embracing 
distance  education  using  online  strategies,  a variety  of  instructional  design  guidelines  and  pedagogical  approaches 
are  being  implemented  to  guide  online  education. 

One  trend  in  online  education  is  the  utilization  of  hybrid  learning  environments,  primarily  at  higher 
education  institutions,  which  offer  a combination  of  online  and  face-to-face  (F2F)  instruction.  The  goal  of  hybrid 
learning  is  to  improve  the  educational  experience  for  students  by  joining  together  the  best  features  of  in-class 
teaching  with  the  best  features  of  online  learning  to  promote  active  independent  learning  and  reduce  class  seat  time 
(Garnham  & Kaleta,  2002;  Young,  2002).  Garnham  and  Kaleta  (2002)  at  the  University  of  Wisconsin-Milwaukee,  a 
leading  institution  in  hybrid  course  development,  assert  that  hybrid  courses  offer  many  advantages  over  F2F  or 
completely  online  courses  including  convenience,  interaction,  flexibility,  and  increased  learning  and  retention. 
Research  on  the  effectiveness  of  online  education  in  general  also  demonstrates  that  students  who  learn  at  a distance 
do  not  learn  any  worse,  or  any  better,  than  traditional  students  (Simonson,  Smaldino,  Albright  & Zvacek,  2003,  pp 
8-9). 

Hybrid  learning  is  not  being  discussed  in  as  much  detail  at  the  K-12  level;  however,  many  high  schools 
such  as  the  Florida  Virtual  School  are  implementing  this  hybrid  approach.  In  fact,  some  educators  believe  hybrid 
learning  could  become  the  norm  in  K-12  settings  (Wood,  2005).  Two  different  approaches  to  K-12  hybrid  learning 
are  being  utilized  within  the  United  States  - 1)  students  taking  courses  in  school  in  a face-to-face  environment  and 
out  of  school  in  an  online  education  course  and  2)  students  enrolling  in  online  education  courses  within  school  while 
teachers  are  facilitators  assisting  when  necessary  and  instructing  lessons  that  enhance  and/or  complement  what  they 
are  learning  online  (Wood,  2005). 

Adventure  learning  (AL)  encapsulates  this  second  approach  to  K-12  hybrid  learning  providing  students 
with  opportunities  to  explore  real-world  issues  through  authentic  learning  experiences  within  collaborative  learning 
environments  (Doering,  2005).  AL  utilizes  both  F2F  and  online  learning  environments  but  is  subtly  different  from 
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traditional  hybrid  environments.  For  example,  in  AL  environments,  classroom  teachers  are  not  positioned  in  the  role 
of  teacher/facilitator/designer  in  the  online  learning  spaces.  AL  online  spaces  are  collaborative  spaces  where 
students,  teachers,  experts,  and  AL  team  members  interact  with  one  another;  these  are  community  spaces  where 
traditional  hierarchical  classroom  roles  are  blurred  and  learning  is  transformed.  Learning  is  transformed  as  students 
no  longer  initially  look  to  their  classroom  teachers  for  knowledge,  but  utilize  the  online  collaboration  zones  to 
interact  and  search  for  answers  while  developing  new  questions  with  other  learners  throughout  the  world. 
Additionally,  student  motivation  to  enter  the  online  learning  environment  is  significantly  enhanced  as  real-time 
features  and  unknown  locations  of  adventure  learning  environments  provide  a hook  for  learners  to  enter  and  return 
to  the  online  environment  frequently  (Doering,  2005).  Furthermore,  students’  roles  also  transform  due  to  the 
flexibility  and  design  of  the  AL  learning  environments  as  they  move  from  student  to  reflective  practitioner  (Palloff 
and  Pratt,  1999).  AL  students  become  more  assertive,  directive,  enthusiastic,  and  motivated  as  they  collaborate, 
construct,  and  learn  with  others  around  the  world  (Doering,  2005). 

As  a result  of  the  changing  roles  of  the  teacher  and  students,  AL  encourages  transformative  learning 
(Mezirow,  1990,  1991),  learning  that  occurs  through  dialogue  based  upon  collaborative  opportunities,  authentic 
experiences,  and  interpretation  of  the  AL  experiences  (Doering,  2005).  Palloff  and  Pratt  (1999)  assert  that 
transformative  learning  is  the  result  of  intense  collaboration  and  reflective  participation  in  a learning  environment; 
transformative  learning  seeks  to  enable  learners  to  understand  why  they  see  the  world  as  they  do  while 
understanding  the  impact  of  prior  knowledge  on  their  newly  constructed  knowledge  (Palloff  and  Pratt,  1999). 

Defining  Adventure  Learning 

Arctic  Transect  2004 

Arctic  Transect  2004:  An  Educational  Exploration  of  Nunavut  (AT  2004)  was  an  adventure  learning  online 
education  program  designed  to  engage  learners  in  authentic  content  through  a real-time  adventure  - a 3,000-mile  dog 
sled  expedition  across  the  newest  territory  in  the  Canadian  Arctic,  Nunavut.  This  online  program  provided  a free, 
eighteen-week,  K-12  curriculum  and  online  education  environment  for  students  and  educators  around  the  world. 

The  Washington  Times  gave  the  AL  site,  www.polarhusky.com,  an  “A”  ('Szadkowski,  2004)  and  the  e-Learning 
Advocate  noted,  “This  is  what  online  learning  is  all  about.  Transporting  students  to  the  Arctic  where  they  can 
experience  a learning  adventure  in  real  time”  (Hall,  2004).  A middle-school  teacher  who  used  AT  2004, 
commenting  on  the  interaction  of  the  “real  world”  with  the  curriculum  in  the  online  learning  environment  stated, 
“This  is  history  in  the  making  and  that  it’s  delivered  to  us  daily  —it’s  just  a phenomenon. . .1  have  never  seen  students 
motivated  to  learn  like  this!  That  is,  where  they  are  motivated  to  go  online  into  the  environment  and  share  their 
experiences  with  other  students  from  around  the  world” 

If  this  is  what  “online  learning  is  all  about,”  what  is  adventure  learning  and  what  does  it  mean  for  distance 
education?  This  paper  describes  this  online  education  approach,  outlines  a framework  for  the  instructional  design 
process  needed  to  develop  an  effective  adventure  learning  education,  and  showcases  an  adventure-learning 
archetype. 

The  term  “adventure  learning”  has  been  used  in  many  instances  to  describe  entirely  different  learning 
opportunities  and  environments.  For  example,  team  building  resources  and  activities  hosted  by  The  Tahoe 
Adventure  Learning  Institute  (http://www.tahoeadventurelearning.com/)  to  “multi-disciplinary”  online  programs 
such  as  The  JASON  Project  (http://www.jasonproject.org)  and  The  Wilderness  Classroom  Organization 
(http://www.wildernessclassroom.com/).  Some  “adventures”  are  programs  that  have  teacher  and  student  lessons 
(The  Blue  Zones,  2006);  others  have  a curriculum  attached  to  the  adventure  with  videos,  online  chats,  and  digital 
labs  (The  Jason  Project,  2006);  while  others  yet  are  simply  an  expedition  masquerading  with  education  as  the  reason 
for  the  human  need  to  explore  the  earth,  but  oddly  put  education  in  the  last  development  phase  of  the  project 
(yourexpedition,  2006).  Across  all  of  these  online  projects,  all  toting  “adventure”  in  their  descriptions,  never  is  there 
a definitive  sense  of  what  adventure  learning  is  nor  mention  of  principles  that  guide  meaningful  learning  in  the 
classroom.  John  Dewey  (1938)  notably  acknowledged,  “the  belief  that  all  genuine  education  comes  about  through 
experience  does  not  mean  that  all  experiences  are  genuinely  or  equally  educative”  (pp  25). 

I define  adventure  learning  (AL)  as  a hybrid  online  educational  environment  that  provides  students  with 
opportunities  to  explore  real-world  issues  through  authentic  learning  experiences  within  collaborative  online 
learning  environments.  Duffy  and  Kirkley  (2004)  state  that  the  quality  of  distance  education,  just  like  any  face-to- 
face  course  or  seminar,  comes  down  to  the  “design  of  and  student’s  engagement  in  the  learning  environment”  (pp  4). 
Researchers  have  identified  approaches  to  develop  online  courses  in  general  (Collis  & Moonen,  2001;  Jolliffe,  Ritter 
& Stevens,  2001)  that  have  provided  guidelines  for  designers;  however,  there  are  few  established  guidelines  for 
designing  and  developing  adventure  learning  distance  education  environments. 
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Adventure  Learning  Framework 

Designers  of  adventure  learning  online  education  environments  may  consider  employing  the  Adventure 
Learning  Framework  developed  from  four  years  of  designing,  developing,  delivering,  and  assessing  adventure 
learning  environments  to  students  and  teachers  throughout  the  world.  This  framework  is  comprised  of  seven 
principles,  which  are  interdependent  of  one  another:  1)  a researched  curriculum  grounded  in  problem  solving,  2) 
collaboration  and  interaction  opportunities  between  students,  experts,  peers,  and  content,  3)  the  utilization  of  the 
Internet  for  curriculum  and  learning  environment  delivery,  4)  the  enhancement  of  curriculum  with  media  and  text 
from  the  field  in  a timely  manner,  5)  synched  learning  opportunities  with  the  AL  curriculum,  6)  pedagogical 
guidelines  of  the  curriculum  and  online  learning  environment,  and  7)  education  that  is  adventure -based  (Figure  1). 

Adventure  Learning  Model 
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Figure  1:  Adventure  Learning  Model 

These  seven  principles,  which  collectively  comprise  the  Adventure  Learning  Framework,  are  expanded 
upon  and  outlined  below  using  examples  from  an  adventure  learning  archetype  (Arctic  Transect,  2004).  These 
principles  can  be  utilized  as  a guide  for  designers  who  wish  to  provide  transformative  learning  opportunities  for 
students  within  an  online  distance  education  experience. 

1.  Adventure  learning  education  begins  with  a researched  curriculum  grounded  in  problem  solving. 

Description.  The  majority  of  curricula  are  not  written  with  AL  in  mind  and  the  majority  of  adventures  or 
expeditions  are  not  developed  with  its  curriculum  in  mind.  For  example,  as  one  researches  many  “adventure 
learning”  online  education  environments,  the  curriculum  that  is  written  does  not  interact  and  correspond  with  the 
activities  within  the  field,  as  the  curricula  are  disparate  from  the  adventure.  Prior  to  planning  any  learning  adventure, 
learning  outcomes  must  be  identified.  From  the  start,  learning  outcomes,  based  on  authentic  problem-solving  tasks, 
should  guide  the  development  of  the  curriculum  and  the  online  learning  environment  (Dexter  et  ah,  in  press)  as  well 
as  the  planning  of  the  adventure.  In  other  words,  AL  should  not  focus  on  the  exploration  or  adventure,  but  rather  on 
the  learning  goals.  The  development  of  curricula  and  online  environments  must  situate  the  learning  in  an  authentic 
environment  knowing  that  the  experiences  are  first  and  foremost  for  educational  purposes,  not  the  thrill  of 
adventure.  Lastly,  the  work  of  cognitive  psychologists  illustrate  that  students  learn  best  when  solving  real-world 
problems  and  the  number  of  instructional  models  related  to  this  learning  approach  reinforce  this  thinking  (Mayer, 
1992).  AL  curricula  are  problem-driven  with  an  overarching  question  that  needs  to  be  solved  through  interacting 
with  the  AL  online  environment.  Students  interact  with  their  peers,  experts,  teachers,  and  curriculum  to  solve  the 
problems  that  are  driving  the  curriculum  modules.  Students  most  often  do  not  look  to  their  teachers  for  answers  as 
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the  problems  being  solved  utilize  information  delivered  from  the  field  as  well  as  the  interactions  with  others  outside 
of  the  classroom.  The  traditional  approach  of  teacher-directed  learning  is  transformed  allowing  students  to  take 
charge  in  the  search  for  answers. 

Example.  Arctic  Transect  2004:  An  Educational  Exploration  of  Nunavut  (AT  2004)  had  two  main  portals,  a 
public  audience  portal  and  a student  and  teacher  portal  - The  Online  Classroom,  which  was  a collaborative  online 
learning  environment  developed  concomitant  with  the  curriculum  to  create  a seamless  learning  experience.  The 
Arctic  expedition  captured  the  interest  of  the  general  public  creating  the  opportunity  to  educate  both  formal 
audiences  within  the  K-12  classroom  as  well  as  informal  audiences.  Therefore,  both  portals  had  the  same  content, 
but  The  Online  Classroom  was  gated  for  the  students’  safety  and  featured  the  collaborative  opportunities.  Because 
students  were  sharing  their  projects  and  their  perspectives  while  interacting  with  peers,  teachers,  and  experts  from 
around  the  world,  the  collaborative  areas  were  gated  with  a username  and  password.  This  filter  process  began  when 
teachers  signed  up  for  the  program  as  their  names,  school,  and  locations  were  confirmed  prior  to  issuing  a username 
and  password. 

The  free  AT2004  300+  page  paper-based  curriculum  was  researched  and  developed  one  year  prior  to  the 
beginning  of  the  project,  and  was  based  upon  the  pre-determined  region  of  travel.  Learning  outcomes  were 
established  taking  into  account  the  grade  levels  that  were  going  to  use  the  program,  the  technology  and  data  that 
would  enhance  the  curriculum,  and  the  online  interactive  activities  supporting  the  paper-based  curriculum.  The 
curriculum  unit  plans  were  developed  in  three  levels  named  Experience,  Explore,  and  Expand  ranging  from  a 
directed  to  a constructivist  approach  to  the  problem-based  content.  The  curriculum  is  written  where  the  students 
must  take  an  active  role  in  solving  the  module  question.  For  example,  one  module  question  was:  “How  are  peoples’ 
actions  in  the  mid-latitudes  impacting  the  life  of  the  Inuit  in  the  Canadian  Arctic  today?”  Only  one  facet  of  the 
answer  to  this  question  could  be  elicited  from  the  curriculum.  A complete  understanding  of  this  question  came  from 
the  numerous  interviews  with  the  Inuit  delivered  to  the  online  learning  environment  from  the  trail  as  well  as  the 
collaboration  opportunities  with  Arctic  experts  the  took  place  within  The  Online  Classroom. 

Teachers  had  ready  and  early  access  to  the  curriculum  giving  them  ample  opportunities  to  work  with  and 
plan  their  teaching  in  advance  of  the  school  year.  The  curriculum  was  made  available  for  download  via  multiple 
portal  document  files  (pdf)  (Figure  2). 

800  PolarHusky.com  / Online  Classroom  {Experience  Authentic  Learning  Online) 
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Building  Nunavut 


Teachers  Guide  - An  introduction  explaining  many 
important  aspects  of  the  Arctic  Transect  2004 
Curriculum  & Activity  Guide. 


Building  Nunavut  - The  word,  "Nunavut",  means 
"our  land"  in  Inuktitut,  the  Inuit  language. 

Nunavut  officially  became  Canada's  newest  territory 
on  April  1,  1999  when  the  Northwest  Territories 
was  divided  into  two.  Since  that  time,  the 
government  of  the  new  territory  has  been  hard  at 
work  responding  to  the  present  and  future  needs, 
hopes,  and  dreams  of  Nunavummiut.  This  unit  will 
explore  the  years  of  negotiations  that  led  up  to  the 
creation  of  the  new  territory  and  the  current 
leadership  and  government  of  Nunavut,  as  well  as 
broader  concepts  of  government  and  leadership. 
Chat  topic:  Defining  Freedom 


Download  Unit  02 
Planning  An  Expedition 


Planning  An  Expedition  - Could  you  carry  your 
house,  food,  cooking  utensils,  clothing  and 
everything  else  that  you  might  need  to  live  and 
survive?  Well,  Arctic  Transect  explorers  do  and  for 
weeks  at  a time,  too!  Experience  the  planning  and 
preparation  required  to  complete  a successful 
expedition  while  choosing  team  members  and 
developing  goal  setting  skills.  Inuit  activities  focus 
on  "Native"  perspectives  and  how  Europeans 
interacted  with  North  America's  original  inhabitants. 
Chat  topic:  The  Effects  of  Exploration 
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People  & Culture  - The  sounds  of  laughter  from  a 
game  of  ajagak  and  the  unique  sounds  of  throat 
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Figure  2:  Curriculum  available  for  download  via  the  Internet 
2.  Adventure  learning  education  provides  collaboration  and  interaction  opportunities  between  students, 
experts,  peers,  and  content. 

Description.  Although  all  AL  education  must  begin  with  a solid  curriculum,  collaboration  is  the  heart  of  the 
educational  experience.  Internet  isolation  should  be  removed  by  providing  collaboration  and  interaction 
opportunities  (Bransford  et  al.,  1999)  which  encourages  transformative  learning  at  multiple  levels- between  students 
and  teachers;  between  students  and  subject  matter  experts;  between  teachers  and  subject  matter  experts;  between 
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students,  teachers,  subject  matter  experts  and  the  adventure  learning  content;  and  lastly,  between  students 
themselves,  teachers  themselves,  and  between  the  subject  matter  experts  (Figure  3).  The  collaborative  and 
interactive  opportunities  were  designed  within  The  Online  Classroom  in  an  environment  named  the  Collaboration 
Zone  (the  term  used  for  the  collaboration  environment  in  the  AT  2004  AL  educational  program). 
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Figure  3:  Adventure  learning  interaction  model 

Student-teacher  and  student-teacher-content  interaction  and  collaboration.  Student-teacher  interaction  and 
collaboration  occurs  primarily  within  the  brick-and-mortar  classroom  as  teachers  utilize  the  curriculum  and  The 
Online  Classroom.  Teachers  guide  students  to  investigate  the  problem  that  needs  to  be  solved  within  the  unit  lesson 
plan  while  using  the  many  media  and  interactive  options  online.  Teachers  are  available  to  answer  student  content 
and  technology-based  questions  while  assisting  them  in  interacting  and  collaborating.  Students  interact  with  the 
curriculum-based  content,  access  the  real-time  content  delivered  from  the  field,  and  collaborate  with  students  from 
around  the  world  within  the  collaboration  zones  in  The  Online  Classroom.  Furthermore,  teachers  are  able  to  access 
content  within  the  curriculum  to  aid  their  content  knowledge  before  and  during  teaching  while  also  accessing  the 
real-time  content  online.  Lastly,  teachers  are  able  to  interact  and  collaborate  with  other  teachers  for  content  and 
pedagogical  knowledge  - both  online  and  within  their  own  school  building. 

Student-expert  and  student-expert-content  interaction  and  collaboration.  Adventure  learning  affords  many 
opportunities  for  students  to  learn  from  subject  matter  experts.  Primarily,  this  occurs  within  moderated  chat 
environments  based  on  the  unit  lesson  plan  topic.  Teachers  scaffold  their  students  on  subject  matter  prior  to  entering 
the  moderated  chat  environments.  Students  submit  their  questions  within  the  chat  environment  to  a moderator  at 
Education  Basecamp  who  chooses  the  questions  that  would  most  enhance  the  learning  discussion.  The  questions  are 
in  turn  submitted  to  the  expert  where  s/he  chooses  what  s/he  wishes  to  answer  before  submitting  it  into  the  chat 
environment.  Thus,  it  is  not  until  the  expert  releases  the  submitted  question(s)  and  answer(s)  that  the  students  see  the 
synchronous  discussion  (Figure  4).  The  collaboration  opportunities  continue  through  asynchronous  discussion  and 
file  sharing  within  collaboration  zones. 
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Figure  4:  Adventure  learning  moderated  chat  & flow 

Teacher-expert  and  teacher-expert-content  interaction  and  collaboration.  K-12  teachers  are  required  to  be 
subject  matter  experts  in  multiple  areas.  Social  studies  teachers  many  times  are  required  to  teach  numerous  courses 
in  areas  that  they  have  not  been  prepared.  Therefore,  the  opportunity  to  collaborate  and  interact  with  practicing 
subject  matter  experts  can  be  a unique  advantage  and  rare  in  classrooms  today.  An  AL  environment  provides  these 
opportunities  through  synchronous  and  asynchronous  discussions  within  the  chat  environments  and  collaboration 
zones  respectively.  The  subject  matter  experts  are  identified  prior  to  the  release  of  the  curriculum  to  the  teachers. 
Thus,  teachers  have  the  opportunity  to  augment  their  content-knowledge  by  interacting  with  the  subject  matter 
experts. 

Student-student,  teacher-teacher,  and  expert-expert  interaction  and  collaboration.  All  AL  participants  - 
students,  teachers,  or  subject  matter  experts  - benefit  from  interaction  and  collaboration  opportunities  with  their 
peers.  Within  AL,  students  work  with  each  other  in  the  classroom,  but  more  importantly,  with  students  from 
throughout  the  world  within  the  chat  and  collaboration  zone  environments.  Students  share  media  (e.g. . .photos, 
videos,  and  Microsoft  Powerpoint™  files)  and  discuss  the  content  through  various  opportunities.  Teachers  normally 
interact  and  collaborate  on  lesson  plan  ideas  and  appropriate  pedagogy,  and  subject  matter  experts  normally  discuss 
their  area  of  research  and  how  to  effectively  communicate  this  with  the  K-12  population. 

Student-content,  teacher-content,  and  expert-content  interaction  and  collaboration.  All  AL  participants 
also  have  numerous  opportunities  to  interact  with  the  content  in  various  formats  ranging  from  the  paper-based 
curriculum  to  the  real-time  updates  delivered  from  the  field.  It  is  important  that  the  teachers  have  the  curriculum  to 
study  the  content  prior  to  the  adventure  for  planning  purposes;  the  subject  matter  experts  must  also  know  their 
discussion  content  topic  weeks  before  entering  the  chat  and  collaboration  environments. 

Typically  within  society,  we  solve  problems  by  working  and  conversing  with  others;  but  despite  research 
that  points  to  collaboration  as  an  important  aspect  of  meaningful  learning  (Jonassen  et  al,  2003)  in  the  classroom, 
students  are  usually  encouraged  to  solve  problems  and  learn  independently.  Adventure  learning  is  based  upon 
creating  opportunities  for  students  to  collaborate  and  reflect  within  the  online  learning  environment  in  order  to 
encourage  transformative  learning.  Transformative  learning  requires  all  learners  to  work  together  through  social 
negotiation  - discussing,  solving,  and  reflecting  on  the  problem  to  be  solved.  Social  negotiation  can  arise  at  high 
levels  with  an  AL  approach  as  individuals  who  are  goal-directed  are  working  together  to  solve  a common  task 
within  a common  place  - a collaboration  zone.  Furthermore,  distributed  cognition  (Vye  et  al.,  1998)  can  occur  when 
students  collaborate  within  an  online  AL  learning  environment  and  research  indicates  that  solving  problems  as  a 
group  is  much  superior  to  individual  problem  solving  (Evans,  1990). 

Example.  AT  2004  provided  collaboration  zones  for  social  negotiation,  which  fostered  problem  solving  and 
motivation  (Doering,  2005).  Collaboration  Zones  provided  learners  with  the  opportunity  to  post  and  share  learning 
objects  while  also  providing  the  opportunity  to  chat  with  subject  matter  experts,  with  expedition  team  members, 
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with  teachers,  or  with  other  students  throughout  the  world.  The  lesson  and  chat  topic  on  global  warming  involved 
students  interacting  with  meteorologist  Dan  Dix  from  The  Weather  Channel™  in  Atlanta,  GA.  Moderated  by 
Education  Basecamp  located  at  the  University  of  Minnesota,  students  were  able  to  ask  Mr.  Dix  questions,  which 
were  then  selected  to  be  answered  by  Mr.  Dix  himself;  the  result  being  a controlled,  synchronous  chat  environment 
(Figure  5). 
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Weekly  chats  are  a premier  component  of  Arctic  Transect  2004  Online  Classroom.  Each 
week  a topic  is  introduced  and  expanded  upon  in  the  Arctic  Transect  2004  Curriculum  & 
Activity  Guide.  Chat  themes  relate  directly  to  each  topic  and  investigate  a current  issue. 
Chats  allow  students  to  talk  about  things  that  are  important  to  them.  Each  chat  is 
facilitated  by  an  Education  Base  Camp  moderator. 


Click  the  above  image  to  enter  the  Weekly  Chat 
on  Tuesday,  May  4 (10am  CST). 
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Download  a pdf  Chat  Tutorial 


Week  17:  Global  Warming  (cont.) 

Background:  The  earth's  atmosphere  is  a complex 
and  dynamic  system  that  protects  and  sustains  all 
life.  Certain  gases  within  the  atmosphere  reflect 
harmful  radiation;  other  compounds  trap  life  giving 
heat  while  other  parts  allow  energy  to  radiate  into 
space.  Greenhouse  gasses  exist  naturally  in  our 


Your  voice  counts!  Share  your 
opinions,  ideas  and  thoughts 
during  weekly  online  chats.  For 
every  week  of  the  expedition,  a 
chat  relates  to  that  week's 
theme.  Talk  about  important 
issues  like  global  warming, 
freedom  and  more. 

Chat  Schedule 


if— n— — w«- 


Po  you  be  Wave  global  warming  Is 
a serious  environmental 
problem*  Why  or  why  not? 


Figure  5:  Expert  chat  schedule  and  directions  with  direct  entry  into  chat  environment 

3.  Adventure  learning  education  utilizes  the  Internet  for  curriculum  and  learning  environment  delivery. 

Description.  Achieving  true  collaboration  and  receiving  timely  updates  from  the  field  requires  the  use  of 
the  Internet.  The  Internet  also  provides  ready  access  to  a curriculum  for  students  and  teachers  throughout  the  world. 
If  one  does  not  utilize  the  Internet  for  an  AL  project,  the  heart  of  AL  - interaction  and  collaboration  with  experts, 
teachers,  and  peers  throughout  the  world  and  timely  and  frequent  updates  from  the  field  - cannot  occur. 
Furthermore,  the  opportunities  for  transformative  learning  are  diminished,  as  students  cannot  collaborate  with  any 
content  or  individuals  outside  of  their  classroom 

Example.  AL  projects  are  able  to  reach  millions  of  students,  offer  collaborative  opportunities,  and  provide 
real-time  experiences  when  the  Internet  is  utilized  effectively  and  efficiently.  Figure  6 showcases  the  “dog  kennel” 
which  was  the  most  frequently  visited  page  of  the  AT2004  3,000+  page  web  site.  Students  would  visit  this  page 
daily  “connecting”  with  their  favorite  Polar  Husky  (there  were  3 1 dogs  on  the  expedition).  Students  would 
consistently  be  motivated  to  sign-in  to  The  Online  Classroom  to  see  what  and  how  their  favorite  dogs  were  doing. 
One  of  the  most  frequently  read  features  of  the  site  was  a daily  update  delivered  from  the  trail  titled,  “Timber 
Tales”,  which  were  the  trail  happenings  from  the  perspective  of  the  dog,  Timber.  The  ability  to  provide  these 
frequent  updates  or  to  provide  collaboration  and  interaction  opportunities  are  based  on  utilizing  the  Internet. 
Furthermore,  educators  participated  in  online  professional  development  activities  that  were  available  to  those 
teachers  who  wished  to  learn  about  AL  teaching  and  learning  approaches  and  received  credit  from  the  university 
without  leaving  their  classrooms  or  homes. 
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Welcome  to  the  Polar  Husky  Kennel.  To  discover  more  information  about  each  Polar 
Husky,  click  on  the  dog’s  photo. 
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Figure  6:  Polar  Husky  dog  Kennel  featuring  bios  of  the  dogs  on  the  expedition 

4.  Adventure  learning  education  enhances  the  curriculum  providing  authenticity  with  media  and  text 
from  the  field  in  a timely  manner. 

Description.  Media  (e.g.,  video,  sound,  QTVR,  and  photos)  and  text  from  the  field  provide  authentic 
enhancements  to  an  AL  curriculum  and  motivation  for  student  learning  (Doering,  2005).  The  media  along  with 
appropriate  pedagogy  motivate  students  as  every  update  from  the  field  reinforces  the  content  of  the  curriculum 
situating  the  learner’s  participation  in  an  authentic  real-time  environment.  Teachers  and  students  must  know  when 
the  field  updates  are  expected  to  be  available  within  the  learning  environment.  Teachers  can  plan  their  lessons 
accordingly  and  also  the  timely  updates  provide  motivation  for  students  entering  the  learning  environment  seeking 
the  latest  adventures  from  the  field. 

Example.  The  AT2004  paper-based  curriculum  was  enhanced  with  frequent  trail  reports  delivered  in  a 
timely  and  predictable  manner.  The  trail  reports,  which  included  movies,  sounds,  photos,  and  interactive  content 
opportunities,  were  delivered  to  the  classrooms  on  a detailed  schedule  (Figure  7).  During  AT2004,  the  expedition 
team  would  not  travel  on  Fridays  as  it  was  coined,  “Education  Day.”  This  non-travel  day  was  used  to  write  the 
weekly  trail  report  and  upload  sound  files,  movies,  and  photos  that  were  available  for  teachers  and  students  at  8:00 
A.M.  every  Monday  morning. 
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TRAII.  UPDATES 

• Weak  17:  Polar  Bear 

• Week  16:  Arctic  Spring 

• Week  15:  Lifted  Spirits 

• week  14:  In  the  Air? 

• Week  13:  H20  wonders 

• Week  12:  In  Search 

• Week  11:  Legendary  Fire 

• Week  10:  Living  Legend 

• Week  9:  Baker  Lake 

• week  B:  Nunavut  Rocks 

• Week  7:  Nunavut 

• Week  6:  Nature 

• Week  5:  Re-supply 

• week  4:  First  Month 

• Week  1:  Lutsel  K'e 

• Week  0:  Preparation 
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» Week  17:  polar  bear  alert;  north  pole?  heat  wave... 


Date:  05.03.04 

Position:  67°40'N  86°42'W,  Nunavut,  Canada 
Weather  Conditions:  Sunny,  breeze  from  north,  28  F/  -2  C 

"IT  IS  A BEAR! !!!!!!!! I! !"...MIIIe  yelled  out. 
"Are  you  serious?"  asked  Aaron,  with  a grin 
on  his  face.  "YYYYES  - Polar  Bear  - POLAR 
BEAR,"  Mille  hollered  frantically  as  she 
realized  just  how  close  it  was.  At  that  same 
moment  Paul  literally  had  his  hand  on  the 
zipper  of  his  tent's  vestibule  door.  Recognizing 
true  fear  in  Mille's  voice,  he  unzipped  and  saw 
it  - yes,  right  there,  seemingly  right  next  to 
Mille  and  Aaron's  tent.  Will  basically  pushed 
him  out  the  door  as  he  rushed  while  yelling,  "Get  out,  get  out;  get  out!" 

A few  minutes  earlier  we  were  jerked  by  very 
sudden,  surprised,  and  angry  warning  barks 
from  Timber,  quickly  joined  by  nervous  barks 
from  Disko.  Mille  lurched  to  the  tent  door,  j,  <3 

ripping  it  open  to  look  out.  "Our  tent  was 
behind  the  sled,  so  all  I could  really  see  were 
the  heads  of  our  dogs  on  the  stake-out,  each 
facing  straight  ahead  with  pointed  ears.  I rose 


Figure  7:  Weekly  trail  reports  delivered  from  the  Arctic 

5.  Adventure  learning  curriculum  provides  synched  learning  opportunities. 

Description.  The  AL  curriculum  is  enhanced  with  a strict  practice  of  providing  synched  instructional 
learning  opportunities.  These  opportunities  begin  with  learning  tools  within  the  online  learning  environment  that 
enable  the  learners  to  interact  live  with  explorers,  educators,  subject  matter  experts,  and  fellow  users.  These  tools 
include  web  offerings  (thematically-based  virtual  tours  and  educational  animated  movies),  multi-media  (access  to 
images,  audio,  video  and  data  from  the  field  that  is  central  to  the  learning  experience),  integration  of  geographic 
information  systems  technology,  collaboration  zones,  and  moderated  chats.  The  online  learning  environment  where 
the  media  opportunities  exist  must  be  designed  in  tandem  with  the  curriculum. 

Example.  Learning  opportunities  must  exist  online  for  students  that  are  synched  with  the  curriculum. 

During  AT2004  when  students  studied  the  unit  entitled  “Water,  Water,  Water,”  the  teachers  and  students  read  the 
newly  posted  trail  report  entitled  “FLO  Wonders”  which  was  synched  with  the  content  of  the  unit  (water).  In  the  trail 
report,  the  students  read  about  the  amount  of  water  the  explorers,  educators,  and  Polar  Fluskies  used  on  the  trail; 
how  the  Inuit  obtain  their  water  in  the  Arctic;  the  impact  of  rising  sea  levels;  and  numerous  other  examples  delivered 
with  movies,  audio  files,  and  photos.  For  example,  photos  and  movies  showing  how  snow  and  ice  were  collected  on 
the  trail  and  melted  for  water  gave  students  an  idea  about  life  on  the  trail.  Scientific  information  such  as  the  impact 
of  a polynia  (open  water  year-round  on  the  Arctic  Ocean)  on  local  weather  phenomena  and  animal  habits  were  also 
documented  as  one  team  member  routinely  collected  snow  samples  for  the  National  Aeronautics  and  Space 
Administration  (NASA),  which  was  also  shared  with  the  AL  users. 

A “local  activity”  was  also  available  in  every  unit  plan  where  students  were  encouraged  to  act  locally  at 
some  level.  During  the  water  unit  plan,  students  recorded  their  water  usage  for  a week  and  a link  was  provided 
within  the  text  from  the  trail  report  that  took  them  immediately  to  a collaboration  zone  to  post  their  own  findings 
(Figure  8). 
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Have  you  ever  thought  about  how  much 
water  you  use  in  a day?  When  Will  did 
research  for  his  book,  "Saving  the  Earth,"  he 
learned  that  the  average  American  would  use 
about  380  liters  (100  gallons)  a day! ! 1 1 1 If 
you  have  a chance,  try  to  record  YOUR  water 
usage  and  add  it  to  Collaboration  Zone  08. 
Water  is  a precious  resource  out  here.  Since  it 
takes  us  so  long  to  prepare,  we  do  not  use  a 
lot  of  water  on  anything  else... be  it  dishes, 
laundry,  or  even  to  wash  ourselves. 


Actually  it  is  not  just  "liquid  water"  but  water 
in  all  its  forms  - be  it  ice  or  snow  - that  is  one 
of  our  most  precious  resources  out  here,  just 
like  it  has  always  been  to  the  Inuit.  Obviously, 
it  is  hard  to  be  dogsledding  without  snow  or 
ice  so  our  need  for  snow  and  ice  might  not 
always  be  quite  as  obvious.  Though  we  do  not 
build  our  "house-on-the-road"  out  of  snow, 
making  igloos  like  the  Inuit  would 
traditionally  do,  we  still  need  ice  or  snow  to 
secure  our  tents  and  anchor  the  dogs  for  the 
night. 


Figure  8:  Trail  report  with  links  to  collaboration  zones  and  media  opportunities 

6.  Adventure  learning  education  has  pedagogical  guidelines  for  the  curriculum  and  online  learning 
environment. 

Description.  The  utmost  value  of  the  AL  environment  is  achieved  when  the  appropriate  pedagogy  is 
defined  and  aligned  with  the  curriculum  and  online  learning  environment.  An  effective  learning  environment  should 
not  be  designed  without  the  instructional  uses  of  it  in  mind.  These  uses  of  the  online  environment,  as  well  as  the 
curriculum,  need  to  be  outlined  in  detail  so  the  curriculum  can  be  implemented  effectively  and  easily. 

Example.  Examples  of  successful  pedagogical  practices  are  vital  to  an  AL  project.  Teachers  must 
understand  how  the  curriculum  was  written  and  how  it  works  together  with  the  online  learning  environment. 

Because  the  paper-based  curriculum  and  features  within  the  AT  2004  learning  environment  were  synched  according 
to  the  experiences  on  the  trail,  a detailed  calendar  was  also  necessary  (Figure  9).  Integration  techniques  were 
available  in  every  unit  lesson  plan  through  screen-capture  videos  and  tutorials  from  experienced  educators  who  have 
successfully  integrated  an  AL  project  within  their  classrooms. 


Figure  9:  “How  to  Use”  Calendar 
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7.  Adventure  learning  education  is  Adventure-Based. 

Description.  AL  education  captivates  and  motivates  learners.  Traveling  to  any  location  on  the  earth 
provides  the  mechanism  to  bring  authentic  content  into  the  classroom.  Because  AL  education  is  delivered  via  the 
Internet,  the  adventure  that  may  be  local  for  one  individual  is  a remote  unknown  distant  location  for  another.  As  we 
strive  to  make  learning  opportunities  more  authentic  so  that  it  is  meaningful  to  students,  the  AL  model  delivers. 

Example.  AT2004  was  based  upon  the  investigation  of  the  Arctic  and  the  traditional  ecological  knowledge 
(knowledge  of  the  environment  that  is  passed  down  from  one  generation  to  another)  of  the  Inuit.  The  slow  traverse 
by  dog  sled  across  Canada’s  newest  territory,  Nunavut,  peaked  student  interest  to  comprehend  the  unknown.  The 
trail  updates  enhanced  the  curriculum  by  providing  authentic  data  in  real-time.  For  example,  when  the  AT2004 
expedition  team  left  Baker  Lake,  Nunavut  and  encountered  an  Inuit  igloo,  photos  and  videos  of  the  igloo  were 
captured  along  with  interviews  with  the  local  Inuit  providing  knowledge  on  how  igloos  were  and  are  still  used 
within  their  environment.  Moreover,  prior  to  arriving  in  Pelly  Bay,  Nunavut,  Inuit  elders  greeted  the  team  on  the  ice 
spending  two  nights  with  the  team.  During  this  time  the  Inuit  hunted  wolves  and  fished  for  Arctic  char  on  the  land. 
The  Inuit  shared  their  Arctic  char  with  the  team  members  as  we  learned  about  this  native  food  and  how  to 
effectively  prepare  it.  The  fish  traps  and  the  exchange  and  preparation  of  the  food  was  documented  and  shared 
online  as  students  studied  the  fauna  of  the  Arctic.  The  unknown  Arctic  continued  to  interest  users  as  the  expedition 
team  crossed  the  Arctic  Ocean,  encountered  polar  bears,  learned  about  global  climate  change,  and  the  loss  of  the 
Inukituk  language. 

The  adventure  for  AL  education  does  not  have  to  be  an  extreme  Arctic  location.  The  education  provided  by 
individuals  sharing  content  from  their  local  environment  such  as  a trip  to  a local  river  investigating  shipping  lanes  or 
a trip  to  a local  crop  farm  will  assist  students  by  providing  authentic  content  that  makes  the  unknown  real. 

Encouraging  Transformative  Learning 

Within  this  paper,  a hybrid  approach  to  K-12  online  education,  adventure  learning,  has  been  defined  and 
described  using  an  AL  design  framework.  An  archetype  of  this  approach,  the  AL  project,  Arctic  Transect  2004,  was 
showcased  identifying  the  features  of  the  program  as  defined  by  AL  principles.  These  principles  support  the  goal  of 
transformative  learning  by  providing  numerous  opportunities  to  collaborate  and  interact  in  real-time  with  real-world 
content,  peers,  teachers,  and  subject  matter  experts  throughout  the  world.  Learners  have  the  opportunity  to  share  and 
discuss  their  perspectives  and  prior  knowledge,  acquire  content  through  a flexible  curriculum,  while  acquiring 
supporting  content  through  multiple  modes  of  media  that  are  grounded  in  authentic  situations. 

Adventure  learning  is  an  exciting  approach  to  hybrid  online  education,  especially  as  the  traditional  field  trip 
is  threatened  because  of  the  lack  of  funds  and  major  focus  on  test  scores  (Standen,  2005).  AL  projects  do  not  need  to 
be  as  extreme  as  crossing  the  Arctic  by  dogsled,  but  can  be  a teacher’s  trip  to  the  local  river  utilizing  the  AL  design 
framework.  As  educators  strive  to  have  students  achieve  meaningful  learning  in  online  learning  environments,  an 
AL  education  gives  students  and  teachers  the  tools  to  create  and  participate  in  learning  that  is  meaningful. 
Transformative  learning  moves  students  from  the  traditional  learner  to  one  who  is  a reflective  practitioner  within 
online  collaboration  spaces  as  well  as  their  brick-and-mortar  classroom. 
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Introduction 

Paris  and  Winograd  (2001)  describe  three  characteristics  of  self-regulated  learning:  (a) 
metacognition,  (b)  strategies,  and  (c)  situated  motivation.  This  paper  combines  theory  and 
research  and  practical  application  examples  to  instructors  using  online  delivery  to  model  and 
scaffold  interactions  which  promote  student  motivation  and  learning  strategies.  Based  on  a study 
of  132  undergraduate  students  using  the  Motivated  Strategies  for  Learning  Questionnaire 
(MSLQ)  and  related  articles  providing  best  practices,  this  presentation  will  help  you  to  improve 
interactions  in  your  online  course.  The  MSLQ  was  developed  by  Pintrich,  Smith,  Garcia  and 
McKeachie  at  the  University  of  Michigan  (1991)  and  consists  of  81  items  and  15  sub-scales  or 
factors  which  can  be  divided  into  a motivation  section  and  learning  strategies/resource 
management  section.  The  MSLQ  was  administered  to  a sample  of  132  undergraduate  students 
and  this  paper  provides  an  analysis  of  their  motivation  and  learning  strategy  levels  accompanied 
by  practical  application  tips  for  teachers  using  online  delivery. 

Results  and  Discussion 

As  shown  in  Table  1,  both  motivation  and  strategy  factors  showed  a normal  distribution. 
While  the  developers  of  the  instrument  avoid  discussion  of  norms,  this  sample  would  seem  to  be 
high  in  the  motivation  factor  with  a mean  value  of  5.3 1 out  of  a possible  7.  Perhaps  even  more 
significant  is  the  lowest  value  of  4.13  which  puts  100%  of  the  sample  above  the  mean  possible 
score  of  3.5.  A variety  of  strategies  are  recommended  to  achieve  best  results  in  improving 
motivation. 

Table  1 

Descriptive  statistics  of  motivation  and  strategy  variables 


Statistic 

Motivation 

Strategies 

Mean 

5.31 

4.34 

Standard  Deviation 

0.49 

0.68 

Skew 

-0.10 

-0.29 

Kurtosis 

-0.33 

.017 

Minimum 

4.13 

2.38 

Maximum 

6.52 

6.10 

The  strategy  variables  also  indicate  a normal  distribution  with  a mean  of  4.34  and  a 
slightly  truncated  range  from  2.38  to  6.10.  While  these  levels  would  not  indicate  a need  for 
concern  about  improving  motivation  and  the  use  of  strategies,  it  is  necessary  for  enhancing 
motivation  for  online  students.  Because  learning,  recall,  and  retention  improve  when  a variety  of 
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strategies  are  employed,  it  is  important  to  know  how  to  improve  motivation  and  strategy  use 
through  online  interactions. 

Review  of  Theory 

MOTIVATION 

Figure  1 illustrates  the  key  factors  of  the  motivation  theory  outlined  by  Keller  in  1987 
which  shows  that  to  be  motivated  a learner  needs:  (a)  to  attend  to  the  information,  (b)  to  find  it 
relevant,  (c)  to  believe  he  or  she  has  the  capacity  to  leam,  and  (d)  to  actually  learn  the  material. 
Because  learners  occasionally  arrive  less  than  fully  motivated,  the  challenge  is  to  implement 
teaching  strategies  that  (a)  arouse  and  sustain  interest,  (b)  connect  with  learner  needs  and 
interests,  (c)  build  and  support  feelings  of  self-efficacy,  and  (d)  support  their  success  and 
satisfaction. 


Figure  1 Keller  ARCS  Model  of  Motivation 

Because  motivation  is  such  a broad  concept,  it  is  helpful  to  focus  in  on  specific  constructs 
for  discussion.  As  shown  in  Figure  2,  Miltiadou  and  Savenye  (2003)  illustrate  constructs  relevant 
to  this  discussion  of  online  learning  motivation.  Specifically  these  are  (a)  the  learner’s  perception 
of  whether  or  not  they  are  able  to  leam  the  material,  (b)  whether  their  goals  are  based  on  ideas 
such  as  performance  or  mastery  and  the  source  of  their  motivation,  and  (c)  whether  or  not  they 
have  the  skills  and  ability  to  evaluate  and  control  their  own  learning.  Supporting  evidence  for 
improving  learner  motivation  comes  from  Lim  and  Kim  (2003)  who  offer  the  following  as 
sources  for  improving  motivation  (a)  reinforcement,  (b)  peer  support,  (c)  relevance,  (d)  interest, 
(e)  self-competence,  and  (f)  affect.  Therefore,  any  activities  that  an  instructor  can  promote  which 
support  these  constructs  will  lead  to  improved  motivation  and  performance  by  the  learners. 

Excellent  advice  on  motivating  instructor  behaviors  comes  from  a book  on  improving 
adult  motivation  by  Wlodkowski  (1999).  Among  those  are  the  need  to  create  a safe  and 
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respectful  learning  environment,  to  have  empathy  for  the  learner,  to  demonstrate  clarity,  offer 
expertise,  and  to  be  enthusiastic.  While  these  are  not  easily  attainable  they  should  still  be  goals  to 
strive  toward. 


Motivational  Constructs 
Miltiadou  & Svenye  (2003) 


Self-efficacy 


Locus  of 
Control 


Attributions 


Goal  Type 


Intrinsic 

Motivation 


Self- 

Regulation 


Figure  2 Miltiadou  and  Savenye  Categories  of  Motivation 

STRATEGIES 

The  fact  that  teaching  is  accomplished  does  not  indicate  that  learning  has  taken  place.  “If 
learners  are  to  be  afforded  the  opportunity  to  regulate  their  own  learning,  then  a metacognitive 
theory  of  learning  has  to  be  developed  in  order  to  redefine  the  outcome  of  learning”  (Singh, 
2005,  p.  1).  One  possibility  is  self-regulated  learning  which  Singh  (2005)  defines  as  “an 
interaction  of. ..metacognition,  motivation,  and  creativity”  (p.  1).  Paris  & Winograd  (2001) 
described  three  ways  that  self-regulation  can  be  taught  including  (a)  explicit  instruction,  (b) 
directed  reflection,  and  (c)  metacognitive  discussions.  Further  guidance  from  McManus  (2000) 
suggests  that  self-regulated  learners  benefit  from  a non-linear  course  organization  and  advance 
organizers  to  activate  prior  knowledge.  Other  strategies  for  promoting  self-regulated  learning 
comes  from  Paris  and  Winograd  (2001)  who  recommend  that  instructors  model  self-regulated 
behaviors  and  support  student  self-regulation  by  having  them  keep  reflective  journals  and  by 
using  frequent  assessment  to  chart  individual  growth.  The  benefits  of  instructional  feedback  on 
achievement  were  further  supported  by  Lim  and  Kim  (2003).  Dobrovolny  (2006)  includes 
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frequent  opportunities  to  assess  and  correct  learning  such  as  questions,  quizzes,  exercises,  and 
simulations  to  promote  enhanced  metacognition. 

Designers  and  instructors  should  “help  learners  personalize  new  information  before, 
during,  and  after  the  instructional  experience”  (Dobrovolny,  2006,  p.  161).  An  important  method 
for  helping  students  to  improve  motivation  and  strategy  is  interaction.  Miltiadou  and  Savenye 
(2003)  added  Learner-Interface  interaction  to  the  traditional  trilogy  of  interaction  which  includes 
(a)  Learner-Content,  (b)  Learner-Learner,  and  (c)  Learner-Instructor.  In  Learner-Instructor 
interaction  it  is  important  to  stimulate  and  maintain  learner  interest,  reinforce  motivation,  assess 
learner  progress,  and  to  provide  support  and  encouragement  to  the  learner. 

Table  2 

Important  teacher  strategies 


Jonassen,  Strobel,  and  Gottdenker  (2005) 

Lim  and  Kim  (2003) 

Teach  knowledge  and  skills  in  real  life,  useful 
contexts 

Ample  opportunities  to  practice 

Provide  new  contexts  for  practicing  new  ideas 

Incorporate  student’s  own  job  examples  in 
authentic  problem-solving  situations 

Engage  students  in  solving  complex,  ill- 
structured  problems  as  well  as  simple  problems 

Check  learning  progress  and  provide  timely 
feedback  and  support 

Collaborative  learning  where  they  socially 
negotiate  a common  understanding 

Interactive  learning  activities 

Help  them  create  mental  models  and  concept 
maps 

Feedback  and  encouragement  emails 

A primary  factor  in  detennining  online  course  quality  is  students’  awareness  of  the 
degree  of  interaction  according  to  a rubric  developed  by  Roblyer  & Ekhaml  (2000).  Interactivity 
in  online  learning  is  equivalent  to  feedback  in  the  traditional  classroom  setting.  Social  rapport 
building  is  also  near  the  top  in  the  list  of  activities  necessary  for  high  quality  course  delivery.  It 
is  a motivating  factor  that  instructors  can  acquire  from  a variety  of  means.  Utilizing  a bio  sheet 
equivalent  of  a syllabus  with  photo  of  the  instructor,  contact  information  and  virtual  office  hours 
is  essential  for  those  online  students  who  are  self-regulated  or  who  have  an  internal  locus  of 
control  (Walters,  2002).  But  students  who  lack  self-regulatory  skills  may  require  instant-on  video 
with  voice  and  in  particular,  easy  navigation  to  proceed.  To  avoid  heavy  download  from  video, 
still  portraits  of  several  poses  with  fade-in  view  take  little  download  time  and  are  very  effective 
particularly  with  instructor  voice  over. 

After  instructors  provide  the  much  needed  social  rapport,  instructional  design 
considerations  for  learning  require  one-way  delivery  of  information  such  as  lectures,  text 
delivery  and  readings.  At  the  end  of  every  one-way  delivery  and  throughout  the  content,  short 
quizzes  or  feedback  pop-ups  should  be  presented  to  maintain  focus,  remediation  and  enrichment. 
These  activities  can  be  presented  in  an  online  accounting  classroom  for  example.  While 
accounting  education  is  transitioning  from  bookkeeping  to  accounting  technology,  the  proper 
mix  of  multimedia  and  interactivity  is  proven  to  be  powerful  teaching  and  learning  technique. 
According  to  Mckee  (2005),  the  only  thing  that  is  holding  accounting  instructors  back  from  truly 
effective  online  instruction  is  a lack  of  creativity. 


121 


A rail  system  module  depicted  by  Mckee  (2005)  provides  students  with  an  opportunity  to 
observe  the  nature  of  cost  accounting  data  extracted  from  the  manufacturing  process.  A 
schematic  was  applied  to  a job-order  cost  system  and  then  a cost  flow  schematic  was 
transformed  into  an  electronic  cost  engine.  The  student  user  can  select  various  buttons  to  confirm 
what  is  going  on  behind  the  scenes  such  as  journal  entries,  transaction  files,  master  files  and  trial 
balances.  The  student  user  can  also  display  animated  clips  that  depict  receiving  raw  material, 
stockroom  requisitions,  time  card  punching,  and  transfers  to  warehouse  and  shipments  to 
customers.  This  can  be  accomplished  by  simply  linking  an  image  to  a page  with  journal  entries 
then  providing  a navigation  button  back.  A quick  video  clip  of  the  proper  way  to  punch  time 
cards  can  easily  be  uploaded  to  your  website  and  linked  accordingly.  The  module  employed 
graphics  and  animation.  Mckee  admitted  the  incorporation  of  lots  of  bells  and  whistles  into  this 
module  was  tempting  but  the  purpose  of  this  module  design  was  to  draw  attention  to  cost 
accounting  systems  and  not  computer-based  accounting  systems. 

The  third  element  of  importance  is  levels  of  interactivity  of  technology  resources.  This 
level  is  where  creativity  helps  instructors  to  design  meaningful  assignments.  An  example  is  a 
program,  Calibrated  Peer  Review  (CPR)  implemented  by  Dungay  and  developed  by  Chapman 
(2004).  This  online  program  allows  students  to  critique  each  other’s  writing.  Students  critique 
sample  essays,  either  authored  by  instructor  or  choose  from  an  assignment  library,  and  the 
computer  keeps  track  of  how  well  they  can  critique.  Their  review  of  another  student’s  review  is 
weighted  by  their  skill  level.  With  one  assignment,  students  leam  about  a topic,  improve  their 
writing,  and  improve  their  reading  skills.  Instructors  are  empowered  to  give  frequent  writing 
assignments  without  increase  grading  workload.  After  submission  of  electronic  essays,  students 
read  and  assign  a score  to  three  “calibration”  essays.  When  students  demonstrate  they  are 
competent  reviewers,  they  read  and  assign  a score  to  three  anonymous  peer  essays,  and  finally,  to 
their  own  essay.  Online,  interactive  assignments  much  like  this  one  will  become  more  available 
in  the  near  future. 

The  fourth  element  is  impact  of  interactive  qualities  as  reflected  in  learner  response.  By 
the  end  of  the  course,  all  students  in  the  class  should  be  interacting  with  instructor  and  other 
students  whether  it  is  through  email,  chatroom,  discussion  board  or  utilization  of  Skype.  Skype 
is  a free  voice  over  Internet  protocol  that  allows  the  ability  to  record  and  play  audio.  The  ability 
to  call  out  and  into  standard  phone  lines  makes  this  a method  of  synchronous  and  asynchronous 
communication  (Descy,  2005). 

Online  learning  is  becoming  an  established  method  of  teaching  and  is  a modern  form  of 
independent  study  that  has  changed  the  way  in  which  the  student  and  teacher  interact 
(Wedemeyer,  1981).  With  the  infusion  of  technology  in  the  mix,  a current  and  more  realistic 
definition  of  online  learning  is  characterized  as  education  or  training  courses  delivered  to  remote 
(off-campus)  locations  (s)  by  means  of  existing  and  evolving  computer  technologies. 

Shifting  from  traditional  to  online  teaching  requires  a paradigm  shift  (Brown,  2004).  In 
general,  navigating  through  course  material  that  has  not  been  meticulously  well  designed  can  be 
difficult  if  not  impossible  for  students.  For  distance  education,  learner-centeredness  is  the  focus 
and  is  strongly  associated  with  the  interface  design  of  an  online  course  (Brown,  1997)  as  well  as 
other  central  design  features.  Learning  management  systems  (LMS)  can  provide  good  design  and 
navigation.  Regardless  of  how  it  is  presented  to  students,  interaction,  immediacy  and  social 
rapport  building  are  essential  to  motivating  student.  Shea,  Motiwalla  and  Lewis  (2001)  found 
that  the  two  most  important  requirements  for  online  student  success  are  immediacy  in  terms 
of  feedback  and  added  student-instructor  interaction. 
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Awareness  of  the  Degree  of  Interaction  (feedback) 

Interactivity  is  one  of  the  main  factors  that  distinguish  online  distance  education  from  the 
previous  forms  of  distance  education  (Magjuka,  Shi  & Bonk,  2005).  New  online  students  may 
not  be  aware  of  the  just  how  they  will  interact  with  their  instructor.  Many  researchers  believe 
that  a higher  level  of  interactivity  between  the  instructor  and  students  correlate  with  higher 
quality  of  the  course.  To  set  the  stage  for  a more  quality  online  learning  environment,  Hohnberg 
(1983)  and  Moore  (1989)  found  in  their  studies  that  student-student  interaction  was  not 
necessarily  significant  to  learning  but  that  instructor-student  interaction  was  an  essential 
component  of  online  instruction. 

What  constitutes  interactivity?  What  is  necessary  for  interactivity  in  online  courses  is 
immediacy  for  infonnation.  The  NET  generation  views  the  world  as  24  hour/7  day  demand  for 
real  time  and  fast  processing  of  information  (Skiba,  Barton  & Amy,  2006).  Feedback  has  become 
more  of  a reactive  than  proactive  task  (Flowers,  2005). 

With  an  emphasis  on  high  levels  of  interactivity  in  distance  education,  problems  usually 
arise  early  in  the  program  with  the  demand  for  immediate  feedback  from  students.  Thus, 
instructors  are  dissuaded  from  teaching  distance  courses  fearing  the  increased  workload  brought 
about  by  the  demand  for  high  levels  of  interactivity  (Hislop  & Atwood,  2000;  Pachnowski  & 
Jurczyk,  2003;  Young,  2002)  and  immediacy.  Roblyer  & Ekhami  (2000)  have  developed  a 
rubric  for  online  interactivity  which  can  be  found  at  http://www.westga.edu/~distance/ 
roblyer32.html. 

Levine  and  Sun  (2002)  contend  that  interactive  and  individualized  pedagogy  is  important 
whether  faculty  are  deeply  involved  in  distance  education  or  continue  to  teach  traditional 
courses.  The  authors  outline  three  barriers  to  distance  education.  First  is  that  higher  education 
lacks  a pedagogy  for  using  the  Internet.  The  second  is  that  the  role  faculty  assumes  in  using 
technology  remains  uncertain,  and  the  third  is  that  faculty  needs  to  know  more  about  “interactive 
and  individualized  pedagogy14  (p.  6). 

Interactive  distance  learning  (IDL)  is  a two-way  video  and  audio  interaction  between  the 
instructor  and  student  at  remote  sites.  IDL,  which  most  closely  simulates  real-time  teaching 
(Dyrud,  2000),  may  be  the  most  appropriate  training  system  for  undergraduates,  K-12  and  those 
who  need  creative  “social-based”  teaching  interfaces. 

Elements  of  Importance 

1 . Awareness  of  the  degree  of  interaction  (feedback) 

2.  Levels  of  Interactivity  of  technology  resources 

3.  Social  rapport  building 

4.  Impact  of  interactive  qualities  as  reflected  in  learner  response 


Levels  of  Interactivity 

Low  level  interactivity 

The  first  level  of  interactivity  is  analogous  to  a lecture.  For  students,  they  only  need  to  be 
able  to  read  online  (e-reading)  and  have  the  ability  to  navigate  the  site.  The  Web  designer  need 
only  to  set  the  pace  of  a presentation  in  multiple  media.  The  "automated  lecturer"  accepts  no 
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input  from  the  audience  except  when  to  turn  the  page  to  the  next  topic  or  previous  topic  (Schlei, 
2000).  In  low  level  interactivity,  the  sequence  of  the  presentation  is  fixed  and  linear.  This  form 
of  interactivity  is  useful  for  teaching  sequential  topics. 

Medium  level  interactivity 

A medium  use  of  interactivity  gives  the  student  (end  user)  the  responsibility  for  choosing 
from  multiple  paths  through  subject  matter.  The  Web  site  responds  to  the  student's  choices 
(clicks)  without  engaging  in  any  dialog.  A multimedia  encyclopedia  provides  this  level  of 
interactivity.  This  level  is  void  of  teaching  but  does  provide  an  atmosphere  where  a student  can 
easily  locate  and  manage  information  (Schlei,  2000). 

High  level  interactivity 

High  levels  of  interactivity  between  instructor  and  student  encompass  responding  to  each 
other  using  electronic  dialog.  Asynchronous  learning  is  not  coordinated  in  terms  of  time  such  as 
synchronous  learning.  This  is  where  high  levels  of  interactivity  are  most  useful. 

Learning  management  systems  (LMS)  such  as  WebCT,  e-College,  etc.),  for  example,  can 
provide  high  levels  of  interactivity  through  its  various  tools  such  as  email,  bulletin  board,  chat 
rooms,  etc.  LMSs  can  determine  the  user's  individualized  path  through  the  Web  site  resulting  in  a 
log  of  every  step  a student  takes  in  the  virtual  classroom.  I my  experience,  those  student  who 
access  the  site  the  most,  generally  receive  a higher  grade  for  the  class. 

An  example  of  high  level  interactivity  is  a simulation  of  an  environment  where  the 
student  is  prompted  to  respond  to  simulated  conditions  with  tools  comparable  to  the  controlling 
factors  of  real  conditions  in  an  actual  environment.  Because  problems  encountered  in  everyday 
situations  are  typically  emergent  and  not  well  defined,  Jonassen  (2003)  recommends  interactivity 
for  learning  by  using  multiple  representations  to  help  students  transfer  their  skills  when  problem- 
solving (Brown,  2004). 

High  levels  of  interactivity  do  not  have  to  be  developed  through  complicated  authoring 
software  such  as  Authorware,  Quest,  Toolbook  and  others.  Representations  can  include  structural 
knowledge,  procedural  knowledge,  reflective  knowledge,  images  and  metaphors  of  the  system  of 
strategic,  social/relational,  conversation/discursive  and  artificial  knowledge.  The  author  suggests 
courseware  with  capabilities  for  posting  discussion,  data  displays  and  threaded  conversations 
support  using  cognitive  tools  to  introduce  and  solve  problems  (Brown,  2004)  can  be  found  in  any 
LMS. 

Other  examples  that  provide  online  interactivity  are  blogging  which  is  a web-log  that 
allows  students  to  contribute  to  and  comment  on  the  blog  entries.  Living  books  help  learners 
work  their  way  through  the  chapters  of  electronic  book  by  providing  directions  to  web  sites  to 
find  information  and  respond  to  questions.  To  view  interactive  chapters  of  The  Net  Generation: 
Implications  for  Nursing  Education  and  Practice,  go  to  www.electronicvision.com/nln/. 

Other  levels  of  interactivity  can  be  obtained  by  using  games  and  simulations.  The  results 
of  research  conducted  by  Evans  (2002)  indicated  that  computer  games  help  develop  problem- 
solving, reasoning,  and  sequencing  skills  for  students  and  that  students  cooperate,  concentrate, 
and  make  a sustained  effort  when  they  are  involved  in  computer  games. 

Nurses  can  master  clinical  skills  in  virtual  settings  by  role-playing  patient  interactions, 
changing  the  race,  gender,  or  ethnicity  of  their  of  their  virtual  doppleganger,  which  are  an  astral 
projection  of  the  human  double  (Simpson,  2006). The  data  collected  from  the  student  can  be  used 
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for  scoring  or  not;  however,  it  helps  the  program  make  judgments  about  what  the  user 
understands  and  doesn't  understand  (Schlei,  2000)  . 

In  their  study,  Amor,  Naicker,  Vincent  & Adams  (1999)  stated  adventure  games  appear 
to  provide  the  best  foundation  for  the  development  of  teaching  resources.  Skills  required  to  play 
adventure  games  identified  by  students  include  logic,  memory,  visualization  and  problem 
solving. 

Games  and  Simulations 

Embedding  JavaScript  games  into  your  Web  site  to  create  high  level  interactivity  for 
online  students  is  easy.  Mind  Guesser  teaches  students  how  to  follow  directions.  Mind  Guesser  is 
JavaScript  free  of  charge  and  can  be  found  on  JavaScriptKit.com  Web  site.  The  site  provides 
simple  two  simple  steps  for  cutting  and  pasting  script.  Always  get  permission  to  use  the 
JavaScript.  Or  the  Memory  Game  is  at  http://www.igor.net/games/memory/that  provides  the 
game  for  any  end-user.  This  is  a great  one  for  building  memory  skills.  Reflexer  Test  is  another 
one  from  JavaScriptKit  that  tests  student’s  response  time.  Trail  Cursor  can  be  found  at 
http://web.coehs.siu.edu/wed/wallace/htm%20pages/fun_page.htm.  This  game  is  designed  to 
teacher  students  how  to  communicate  accurately.  Build  your  own  game  using  PowerPoint  or  go 
to  this  Web  site  http://teach.fcps.net/trtl4/Power%20Point%20Games/power_point_games.htm 
to  find  more. Generators  to  help  build  scripts  are  plentiful  on  the  Web.  Lizardpoint.com  and 
Myspace-Generator  are  examples. 

Rule-based  simulations  developed  by  using  computer-based  training  software  such  as 
Quest,  Toolbook  and  Authorware  will  request  some  training;  however,  if  you  need  to  teach 
student  performance  skills,  these  interactive  modules  can  be  designed  to  teach  skills  with  lots  of 
interactivity.  For  example,  modules  designed  using  Authorware  (Figure  3)  can  teach  gardeners 
when  to  plant  certain  varieties  of  plants.  The  scene  is  a sketch  of  a garden  with  four  spots  ready 
for  planting.  The  user  is  instructed  to  drag  the  plants  to  the  plan  so  that  each  is  in  it  optimal 
growing  location.  Hosta’s  should  be  planted  in  a shady  spot,  the  user  would  click  and  drag  the 
image  of  the  Hosta  to  spot  under  the  Silver  Maple,  and  so  on.  If  the  user  makes  a mistake,  the 
image  dings  and  automatically  moves  back  to  its  original  spot. 
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Figure  3 Authorware  module 

Social  Rapport  Building 

Building  social  rapport  involves  the  forging  of  trust  and  respect  among  online  students 
(Kanuka,  Collett  & Caswell,  2002).  When  instructors  begin  their  online  course  with  a welcoming 
email,  this  sets  the  tone  for  the  entire  course.  A still  photo  with  an  audio  message  with  closed 
caption  (marquee  or  scrolling  text)  provides  more  kinesthetic  welcome  to  students.  Since  online 
students  rely  exclusively  on  what  the  instructor  provides  online,  due  dates,  assignments,  quizzes, 
and  any  promises  made,  it  important  that  the  instructor  adhere  to  dates  and  promises  given  (Sull, 
2006). 


Promptness  to  emails  on  the  instructors’  part  is  essential  to  social  rapport  building. 
Regardless  of  the  content  of  the  email  from  students,  a timely  reply  is  mandatory.  This 
acknowledgement  goes  a long  way  to  earning  student  respect.  Send  general  emails  throughout 
the  course.  In  the  traditional  classroom,  eye  contact,  voice  inflections,  etc.  set  the  social  stage  of 
rapport  building.  Online  students  must  be  made  aware  that  someone  cares  about  their  success  in 
the  online  classroom;  immediacy  to  email  will  provide  rapport  similar  to  the  face-to-face 
classroom. 
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Quality  feedback  equates  to  student  motivation.  Give  students  as  an  entire  class  and 
individually  compliment  them  on  their  work  and  what  they  did  wrong  or  did  right.  Use  casual 
language  which  translates  into  a mix  of  conversation  and  fonnal  writing.  In  this  way  students 
will  not  feel  like  the  instructor  is  someone  who  thinks  he/she  is  better  and  above  the  students 
(Sull,  2006).  Sull  suggests  the  use  chat  rooms,  threaded  discussion  boards  and  e-journaling  so 
that  students  can  be  spontaneous  and  be  involved  which  allows  for  more  ownership  of  the  course 
on  their  part. 

The  ideal  online  classroom  might  consist  of  the  student  sitting  with  a book,  pad  of  paper, 
and  PowerPoint  slides  which  are  designed  so  that  they  can  work  along  hitting  the  space  bar  or 
mouse  and  unveiling  the  next  step  or  answer  to  a problem  (Sull,  2006). 

Communication  methods  that  students  enjoy 

• Skype  has  voice  over  internet  protocol  (VoIP)  or  softphone(  Descy,  2005).  It  is  free,  it 
has  chat,  and  it  has  instant  messaging. 

• Blogging  — According  to  Miles  (2006),  a rich  environment  can  emerge  if  careful 
introduction  to  blogging  has  been  established.  With  sufficient  guidance  offered  through 
class  time,  participation  and  simple  technical  support,  scholarly  bloggers  can  collaborate 
with  much  academic  freedom.  "Blog  entries  can  range  from  three-word  bursts  of  sarcasm 
to  carefully  honed  5, 000- word  treatises.  The  sweet  spot  lies  somewhere  in  between, 
where  scholars  tackle  serious  questions  in  a loose-limbed,  vernacular  mode"  (Glenn, 
2003,14). 

• Text  messaging  — compress  a few  words  to  get  a message  to  someone.  It  is  powerful  to 
pick  up  a cell  phone  then  read  a message  on  the  screen. 

• Instant  messaging  — send  messages  to  all  of  the  people  in  your  buddy  list  as  long  as  that 
person  is  online.  Both  people  can  type  in  messages  that  both  can  see. 

• YouTube  — upload,  view,  and  share  video  to  YouTube  for  all  to  view. 

The  only  thing  holding  back  educators  is  creativity. 
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Abstract 

Since  diplomas  are  awarded  based  on  the  demonstration  of  knowledge,  skills,  and  abilities  through  assessment  (not 
on  accumulated  credits  through  time  spent)  at  a competency-based  university,  how  do  adult  learners  become 
competent?  What  is  the  role  of  distance  learning  resources  (online  courses,  e-learning,  library,  and  bookstore 
services)  within  this  model?  This  article  presents  distance  learning  resource  philosophy,  procedures,  and  issues  that 
arise  when  providing  instruction  at  the  US’s  premiere,  accredited,  competency-based  distance  university. 

Introduction 

Imagine  earning  a college  degree  from  the  convenience  of  your  own  home  or  office.  Sounds  like  online  distance 
education,  right?  However,  what  if  you  didn’t  have  to  take  a set  number  of  courses  and  amass  credits,  but  instead 
you  could  demonstrate  what  you  already  know  and  can  do  through  assessments  - affirming  that  prior  learning  and 
ensuring  all  new  learning  would  be  fresh  and  new.  Add  to  that  the  benefit  of  a of  a faculty  member  who  works 
closely  with  you  throughout  your  degree,  and  an  individualized  program  of  study,  tailored  to  your  background  and 
eagerness  to  progress  rapidly  toward  completion.  Couple  that  with  peer  interaction  throughout  your  studies  with 
other  students  and  faculty  members  in  learning  communities.  Suppose  that  you  could  acquire  new  learning  through 
online  courses,  where  you  might  feel  more  comfortable  in  unfamiliar  subject  matter  with  the  structure  and  expert 
guidance  and  feedback  from  an  instructor  - or,  could  have  the  flexibility  of  doing  guided  self-study  of  textbooks  or 
e-learning  modules  for  content  in  which  you  were  familiar  and  only  needed  brush  up  before  assessment.  Finally, 
imagine  that  your  diploma  was  from  a university  that  was  both  regionally  and  nationally  accredited,  refined  to  meet 
the  stated  needs  of  major  employers,  associations,  state  and  national  standards,  as  well  as  industry  certifying  bodies. 
Does  this  sound  intriguing?  Such  is  the  exceptional  educational  model  being  implemented  by  Western  Governors 
University. 

In  1998  the  governors  of  19  states  founded  Western  Governors  University  (WGU)  to  provide  a competency-based 
online  alternative  to  the  campus-based  post-secondary  educational  enterprises  in  their  states.  Although  the 
University  had  the  backing  of  over  20  prominent  corporations  and  foundations,  at  the  time  of  accreditation  by  the 
Distance  Education  and  Research  Council  (DETC)  in  2002,  the  offerings  were  limited  to  a handful  of  bachelor’s  and 
associate  degrees,  and  enrollments  had  reached  a plateau  at  roughly  200  students.  The  receipt  of  unprecedented 
regional  accreditation  by  four  western  commissions  and  grant  support  and  funding  by  the  US  Department  of 
Education  in  2003  turned  the  corner  for  this  innovative  institution.  As  of  September  2006  WGU  is  realizing  its 
original  potential  by  providing  distance  education  to  over  6,100  students  seeking  35  bachelor’s  and  master’s  degrees 
in  Teacher  Education  (many  including  initial  licensure).  Business,  and  Information  Technology,  (with  Health 
Profession  degrees  being  added  in  late  2006).  The  University  has  over  1250  graduates. 

WGU’s  mission  “is  to  improve  quality  and  expand  access  to  post-secondary  educational  opportunities  by  providing 
a means  for  individuals  to  learn  independent  of  time  and  place  and  to  earn  competency-based  degrees  and  other 
credentials  that  are  credible  to  both  academic  institutions  and  employers.”  At  the  heart  of  the  institution  is 
“competency-based  education.”  This  commitment  to  CBE  translates  in  into  all  University  policies,  practices,  and 
procedures,  such  as  mentoring,  use  of  third-party  learning  resources,  and  assessments. 

Throughout  their  degree  program,  candidates  demonstrate  their  competencies  through  a series  of  15  to  25 
substantive  assessments.  To  ascertain  students’  proficiency,  WGU  adheres  to  rigorous  assessment  procedures  to 
measure  competency.  This  includes  proctored  objective  and  essay  exams  (delivered  at  testing  sites  through  contract 
arrangements  throughout  the  country  and  overseas  - for  active-duty  military  personnel),  performance  tasks,  portfolio 
items,  and  capstone  projects.  The  University  uses  both  internally-developed  assessments  and  externally-developed 
ones.  The  latter  consist  primarily  of  industry-recognized  certification  exams,  such  as  the  SHRM  for  human  resource 
management,  or  widely-used  and  validated  teacher  education  assessments,  such  as  the  Praxis  examinations  for 
secondary  education  content  knowledge.  Only  the  demonstration  of  competencies  through  assessment  counts 
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toward  academic  progress  and  degree  completion.  Program  councils,  comprised  of  national  experts  from  the 
academy,  business  and  industry,  government,  and  school  systems,  oversee  the  development  of  WGU  competencies. 
These  senior  academics  consult  for  the  University  on  all  aspects  of  the  academic  programs  and  serve  as  our  most 
senior  faculty  members.  They  typically  meet  in  person  at  WGU  offices  four  times  a year  to  monitor  program 
effectiveness,  evaluate  learning  resources,  and  provide  advice  on  program  planning  issues. 

This  competency -based  approach  allows  affirmation  of  prior  college-level  learning,  while  integrating  it  with  new 
learning.  Not  surprisingly,  WGU’s  average  student  is  around  40  years  old,  working  full-time,  and  hoping  to 
accelerate  through  WGU’s  degree  program  because  of  relevant  experience  before  and  during  the  degree  program. 
Students  are  drawn  to  the  University  for  its  convenience,  as  well  as  its  relevance;  for  instance,  all  study  is  done  at  a 
distance,  and  students  can  start  any  month  of  the  year  in  one  of  WGU’s  six-month  terms.  WGU  courses  of  study 
include:  (1)  mentoring  by  faculty  experts  who  assist  student  work  throughout  their  programs;  (2)  learning  resources 
like  study  guides,  textbooks,  Web  sites,  and  full  online  courses,  that  students  take  to  brush  up  or  build  competencies 
from  scratch  in  some  areas;  (3)  learning  communities  for  peer  interaction  and  learning  under  mentor  guidance;  and 
culminate  in  (4)  assessments  where  students  demonstrate  competency  for  completing  degree  requirements. 

Purpose 

Because  of  the  University’s  competency-based  education  (CBE)  approach,  faculty  roles,  distance  program  delivery, 
and  the  provision  of  learning  resources  is  quite  different  than  at  other  higher  education  institutions.  The  remainder 
of  this  article  focuses  on  the  implications  of  CBE  for  learning  resources  (LR).  It  details  the  role  that  these  LRs  play, 
the  University’s  philosophy  about  their  use,  and  practices  WGU  follows  in  collaborating  with  other  institutions. 

This  information  can  benefit  you,  the  reader,  in  several  ways.  You  will  gain  insight  into  how  this  innovative  model 
of  higher  distance  education  has  been  successfully  implemented  over  the  past  decade.  The  University’s  policies, 
practices,  and  procedures  can  inform  similar  initiatives  within  the  agencies,  institutions,  and  organizations  where 
you  are  involved  with  distance  education.  Additionally,  since  the  CBE  approach  to  learning  resources  is 
collaborative,  this  article  may  give  you  ideas  to  effectively  link  your  own  distance  offerings  to  similar  consortia  to 
create  synergy  for  your  own  distance  education  enterprise. 

WGU’s  learning  resource  brokering  model 

From  its  inception  Western  Governors  University  (WGU)  has  acquired  the  learning  resources  (LRs)  that  it  provides 
its  students  through  licensing  agreements  with  third  parties.  WGU  uses  the  term  “learning  resources”  to  refer  to  the 
wide  variety  of  instructional  supports  that  students  use  to  gain  competency.  These  include  online  instructor-led 
courses  and  independent  learning  resources  (ILRs),  such  as  web  accessible  e-learning  modules,  CD  ROMs,  Videos 
and  workbooks.  LRs  also  encompass  textbooks,  and  library  materials.  The  primary  purpose  of  LRs  is  to  enable 
students  to  develop  the  skills  and  knowledge  necessary  for  them  to  successfully  demonstrate  competency  through 
assessment. 

WGU  seeks  LRs  that  fit  its  model  of  competency-based  distance  education.  These  LRs  need  to  be: 

• independent  of  time  and  place  - since  WGU  students  are  located  through  the  country  as  well  as  some  stationed 
overseas; 

• scalable  - since  there  are  hundreds  of  students  in  each  degree  program  with  likely  increases  throughout  the 
foreseeable  future; 

• affordable  - since  LR  costs  are  paid  through  WGUs  own  low-cost  general  tuition,  with  70  percent  of  WGU 
students  receiving  financial  aid; 

• available  for  open  or  frequent  enrollment  - since  WGU  students  start  monthly  and  are  on  individual  six-month 
terms; 

• modularized  - since  WGU  recognizes  that  prior  competencies  shouldn’t  require  students  to  repeat  material  they 
already  know; 

• feedback  affording  - since  students  require  response  about  their  capability  and  readiness  to  assess;  and 

• self-paced  - since  students  need  to  be  able  to  progress  according  to  their  schedule;  particularly,  WGU  seeks 
LRs  that  allow  students  to  accelerate. 


132 


WGU’s  LRs  come  from  accredited  higher  education  institutions,  like  Rio  Salado  Community  College,  Chadron 
State  University,  and  Davenport  University,  as  well  as  from  commercial  educational  enterprises,  such  as  American 
Museum  of  Natural  History,  Wasatch  E-learning,  Canter  and  Associates,  Thomson  NetG,  and  Abromitis  Online. 
WGU  seeks  the  highest  quality  LRs  that  fit  its  model,  and  retain  the  flexibility  to  use  similar  LRs  covering  the  same 
content  and  swap  out  LRs  as  it  identifies  better  ones. 

WGU  faculty  members,  known  as  mentors,  assist  their  students  in  taking  pre-assessments  that  identify  individual 
areas  of  strength  and  weakness.  The  mentor  works  with  each  student  to  build  an  online  academic  action  plan  that 
schedules  the  assessments  that  they  will  complete  during  the  term  with  appropriate  learning  resources.  The  possible 
learning  resources  include  two  types:  (1)  guided  independent  study  LRs  — for  students  with  competency  in  the 
domain  and  who  are  self  directed;  (2)  more  structured  LRs  - usually  instructor-led  courses,  for  students  with  little  or 
no  competency  in  the  domain  and  who  with  limited  self  direction.  Students  enroll  in  their  LRs  directly  through  their 
online  academic  action  plan,  allow  them  access  within  minutes  to  web-based  e-learning  LRs,  triggering  same-day 
shipment  of  materials,  and/or  getting  them  instructions  to  enroll  in  their  selected  term-based  online  course  within 
days. 

WGU  seeks,  as  much  as  possible,  to  identify  existing  resources  that  align  closely  to  its  competencies  and 
performance  tasks  and  use  them  “as  is.”  When  off-the-shelf  LRs  cannot  be  found,  WGU  works  with  its  providers  to 
first  delivery  modifications  - packaging  the  LR  into  smaller  units  or  larger  modules  (courses)  or  creating  open- 
enrollment  or  staggered  offerings,  fitting  enrollment  demands.  If  actual  course  content  and  learning  activities  need 
to  be  addressed,  it  will  do  so  by  asking  EPs  to  perform  content  customization  - inserting  or  elimination  of 
objectives,  content,  and  activities.  In  rare  cases  WGU  asks  a provider  to  build  LRs  to  WGU  specifications  with 
WGU  academic  personnel  providing  occasional  input  with  content  expertise,  assessment  suggestions,  delivery 
requirements,  and  learner  characteristics. 

Although  adult  students  come  to  WGU  expecting  to  take  online  courses  (chiefly  reflecting  their  former  experience 
with  higher  education),  increasingly  the  University  is  moving  toward  offering  independent  learning  resources  (ILRs) 
supported  within  its  learning  communities.  WGU’s  goal  is  for  students  to  become  independent  self-directed 
learners.  However,  third-party  online  courses  too  often  foster  reliance  on  course  pacing,  structure,  content,  and 
professor  direction  that  may  not  meet  individual  student  needs.  Also,  the  University’s  institutional  research 
indicates  that  those  students  who  dropped  out  preferred  courses,  whereas  those  who  progressed  and  graduated 
preferred  ILRs.  Learning  communities  provide  similar  peer  discussion  and  expert  feedback  from  community  leaders 
as  their  course  counterparts  do,  but  the  instructional  content  comes  through  e-learning  modules,  online  articles,  Web 
site  links,  and  other  learning  objects  with  which  students  engage.  Students  can  independently  interact  with  the 
learning  assets  of  these  learning  communities  - at  their  own  pace,  structure,  and  according  to  their  own  competency- 
development  needs.  Similarly,  WGU  wants  to  bring  the  various  functions  of  its  online  library  services  (described 
below)  within  these  learning  communities  with  enhanced  interfaces  to  specific  content.  In  the  future  it  also  wants  to 
make  relevant  portions  of  its  adopted  textbooks  available  within  these  communities  in  electronic  format  as  well. 

Library  and  bookstore  services 

WGU’s  competency-based  approach  enables  student  independent  study  under  guidance  of  a mentor,  following  study 
guides,  and  aided  by  the  peer  support  found  in  learning  communities.  Since  its  opening  in  1998  the  University  has 
contracted  with  the  University  of  New  Mexico  for  library  services  to  support  all  academic  areas.  A librarian 
provides  the  primary  interface  with  UNM  library  resources  for  reference  services,  database  search  and  full-text 
resource  acquisition,  interlibrary  loan  services,  and  for  electronic  reserves.  Orientation  to  this  distance  library  is  an 
integral  part  of  a mandatory  course  in  which  students  participate  when  they  join  WGU.  Through  early  research 
projects,  they  become  familiar  with  using  library  resources.  Similarly,  WGU  contracts  with  an  online  library  vendor 
to  keep  a constant  supply  of  its  primary  textbooks  on  hand.  The  bookstore  sells  these  at  discounted  prices  to 
students,  while  providing  reliable  shipping  and  buy-back  services. 

The  LR  model  in  action 

The  following  depiction  of  a fictitious  student,  Laura,  represents  the  detailed  learning  experience  in  the  University’s 
business  degree  program.  Laura  works  in  the  personnel  department  of  a major  corporation.  She  advanced  quickly 
in  her  job  during  early  years  with  the  company  based  on  ability  to  quickly  pick  up  necessary  skills  and  a natural 
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knack  for  working  with  various  types  of  employees.  A year  ago,  Laura  was  chosen  to  work  closely  with  the  human 
resource  manager,  shouldering  many  of  the  responsible  duties  herself  when  there  was  a department  crisis.  However, 
recently  she  has  felt  stifled  recently  because  promotions  have  gone  to  those  with  bachelor’s  degrees,  and  rather  than 
doing  the  human  resource  management  tasks  of  which  she  is  capable,  she  has  been  relegated  to  the  more  routine  and 
clerical  tasks.  Recognizing  her  need  for  a bachelor’s  degree  in  human  resource  management  to  move  ahead,  Laura 
confronted  the  difficulty  of  trying  to  “go  back”  to  school  - now  married  with  an  infant  son  and  need  for  full-time 
employment  to  pay  the  bills.  Her  husband  and  Laura  talked  about  how  she  might  capitalize  on  the  several  semesters 
of  college  she  had  right  after  high  school,  and  possibly  incorporate  some  of  the  human  resources  training  and 
experience  while  she  has  worked  in  this  field  over  the  past  seven  years.  When  they  discovered  WGU,  it  seemed  like 
an  answer  to  prayer. 

After  visiting  the  WGU  website  and  expressing  interest  through  an  online  form,  she  appreciated  the  personalized 
attention  of  an  enrollment  counselor  who  addressed  her  concerns  and  answered  her  questions  in  a telephone  call  and 
exchange  of  emails.  An  inquiry  session  via  telephone  conference  with  a business  faculty  member  (mentor)  and  a 
group  of  other  prospective  students  answered  her  questions  satisfactorily.  She  was  pleased  to  be  able  to  start  within 
weeks  of  applying,  since  the  introductory  course,  “Education  Without  Boundaries,”  for  new  students,  starts  every 
month.  During  this  course  she  learned  how  to  juggle  her  time  to  fit  in  the  15  to  20  hours  per  week  demanded  by  the 
program,  learned  to  participate  in  learning  communities,  and  learned  to  use  the  University’s  bookstore  and  library. 
Her  first  writing  assignment,  a mini-research  paper  on  a business  topic,  required  that  she  pull  several  full-text 
articles  from  the  online  library’s  databases,  and  use  them  correctly,  following  APA  format  (which  was  new  to  her). 
During  that  first  month  Laura  took  pre-assessments  to  determine  her  content  strengths  and  abilities.  She  also  met 
her  mentor  by  phone,  they  shared  each  others’  backgrounds,  and  together  developed  an  academic  action  plan  (AAP) 
- based  on  transcript  evaluations,  pre-assessment  scores,  and  Laura’s  commitment  to  work  hard  and  progress  rapidly 
through  the  program.  Her  mentor  inspired  confidence  in  Laura’s  decision  to  get  a Bachelor  of  Science  degree  in 
Business  with  a Human  Resources  Management  emphasis,  and  Laura  felt  the  mentor  had  the  skill,  empathy,  and 
expertise  to  guide  her  through. 

Starting  with  liberal  arts,  Laura  centered  her  studies  in  that  first  six  months  on  completing  the  Language  and 
Communications  (L&C)  Domain  and  Quantitative  Literacy  Domain  assessments.  It  had  been  quite  some  time  since 
Laura  had  written  formally,  so  she  enrolled  in  an  eight-week  instructor-led  Composition  course  taught  by  one  of 
Abromitis  Online’s  instructors  (an  education  provider  with  whom  WGU  contracts).  The  online  course  followed  an 
ambitious  syllabus,  moving  through  the  mechanics  of  writing  (e.g.,  grammar),  expository  essay  writing,  to  even 
developing  a 10  page  research  paper  (getting  references  from  the  WGU  library).  While  taking  the  course  (and 
immediately  afterward),  Laura  worked  with  WGU  to  schedule  herself  for  assessments,  a comprehensive  objective 
exam  and  essay  exam  (which  were  taken  at  a proctored  testing  center  of  a local  community  college  near  her  home), 
and  she  was  able  to  turn  in  the  research  paper  (with  modifications  suggested  by  her  mentor)  to  pass  the  performance 
task  requirement  of  the  L&C  domain.  Math  had  always  been  Laura’s  strong  suit,  so  she  opted  to  self-study  the 
competency  requirements,  using  the  WGU  study  guide,  the  recommended  textbooks  which  she  purchased  from 
WGU’s  online  bookstore  and  a self-paced  “Thinkwell’s  Intermediate  Algebra”  package  of  CDs  and  Website  e- 
learning;  the  package  arrived  in  the  mail,  within  five  days  of  enrolling  in  this  LR  through  her  AAP.  Laura  enjoyed 
the  QL  assessments  - a proctored  objective  examination  taken  by  computer  at  the  same  testing  center,  covering 
algebra,  probability,  and  statistics,  as  well  as  a more  involved  performance  task  - working  with  an  Excel  spreadsheet 
to  demonstrate  working  knowledge  of  home  financing,  analyzing  amortizations  covering  several  scenarios. 

The  next  few  six-month  terms  allowed  Laura  to  “get  down  to  business”  as  she  addressed  the  lower-level  Business 
Core  while  also  completing  other  liberal  arts  domains  like  Natural  Sciences  and  Visual  & Performance  Arts.  Much 
of  the  business  core  involved  “brash  up”  work,  using  the  study  guides,  textbooks  that  she  ordered  from  WGU’s 
online  bookstore.  She  also  used  e-learning  modules  in  various  business  topics  such  as  in  accounting,  law, 
economics,  marketing,  management,  ethics,  and  information  technology  from  the  Thomson  NetG  library,  which  had 
been  tailored  to  fit  WGU’s  business  competencies.  Laura  particularly  enjoyed  these  studies  because  she  could  relate 
these  concepts  to  the  multifaceted  corporation  in  which  she  worked.  The  connections  were  remarkable!  The  liberal 
arts  enabled  Laura  to  study  new  areas  which  she  only  had  minimal  exposure  to  in  high  school.  This  refreshed  that 
knowledge  by  Laura  taking  online  classes,  engaging  with  CD  ROMs  and  websites  (e.g.,  Thinkwell’s  Science 
program).  Those  were  fascinating  at  first,  but  Laura  breathed  a sigh  of  relief  when  she  passed  the  associated 
assessments  in  those  areas.  An  intriguing  assessment,  required  of  all  bachelor’s  degree  students,  is  Collegiate-level 
Reasoning  and  Problem-solving  Skills  (CLRPS).  A course  from  Abromitis  Online  which  prepared  Laura  for  this 
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assessment,  involved  researching  a contemporary  business  topic  from  several  perspectives,  and  writing  a 
comprehensive  essay  explaining  those  sides  as  well  as  arguing  for  one’s  own  position.  Laura  chose  to  examine 
positions  to  make  US  companies  and  executives  act  ethically.  Following  the  course  study  guide,  she  accessed  some 
articles  placed  on  e-reserve  at  the  WGU  online  library  about  her  specific  question,  and  found  related  full-text  articles 
from  the  library’s  online  ABI/Inform  database  to  buttress  the  various  viewpoints  and  her  own  perspective.  She 
received  great  satisfaction  from  the  feedback  she  received  as  she  progressively  completed  the  various  CLRPS 
assignments,  and  was  pleased  when  her  final  essay,  which  she  turned  in  to  WGU  assessment,  passed  the 
requirements. 

Laura’s  final  three  (six-month)  terms  were  devoted  to  upper-division  business  studies  in  the  domains  of  Leadership 
and  Professionalism,  Business  and  Technical  Knowledge,  and  Fluman  Resource  Management.  Tier  managers  at 
work  recognized  the  additional  knowledge  and  skills  that  she  was  learning  in  her  business  program,  and  Laura  found 
herself  with  opportunities  to  use  these  in  additional  and  more  responsible  duties  on  the  job.  Doors  were  starting  to 
open  at  work,  even  though  she  had  almost  a year  to  graduate.  Learning  resources  for  these  domains  include 
textbooks,  the  NetG  e-learning  modules,  as  well  as  some  targeted  online  courses  from  WGU’s  education  providers. 
To  get  the  degree  with  an  emphasis  in  human  resource  management,  Laura  had  to  pass  the  rigorous  Professional  in 
Fluman  Resources  (PF1R)  certification  exam,  developed  and  administered  by  the  Fluman  Resource  Certification 
Institute  or  the  Society  for  Fluman  Resource  Management.  For  this  high  stakes  assessment,  Laura  spent  over  three 
months  preparing  diligently  in  an  online  course  for  that  specific  purpose,  offered  through  WGU’s  provider  contract 
with  Davenport  University.  Not  only  did  she  interact  with  the  instructor  and  other  students  through  online 
discussions,  there  was  a certification  preparation  package  that  came  with  the  course  - textbooks,  workbooks,  and 
access  to  practice  exams  at  a website.  Laura  was  thrilled  to  finally  pass  that  assessment  and  receive  the  industry 
standard,  PFIR  certification;  this  is  an  important  credential  in  addition  to  the  WGU  diploma  that  would  further  her 
career  within  her  company  or  elsewhere. 

The  final  requirement  in  Laura’s  degree  took  most  of  her  final  semester:  to  compile  a portfolio  and  develop  a 
capstone  project  that  demonstrated  the  breadth  of  her  learning  as  well  as  the  integration  of  knowledge  and  skill.  For 
the  portfolio  the  Laura  wrote  a reflective  essay  integrating  the  work  she  had  done  throughout  her  degree  on  a 
decision  she  made,  revisiting  that  decision  in  light  of  who  she  had  become.  Additionally,  for  the  capstone  Laura 
completed  a major  project  that  integrated  and  synthesized  competencies  in  all  domains  required  for  the  bachelor  in 
business  degree;  she  demonstrated  various  competencies  in  language  and  communication,  quantitative  literacy, 
leadership  and  professionalism,  and  the  several  business  areas  of  management,  marketing,  economics,  ethics  and 
professionalism,  along  with  her  major  of  human  resource  management. 

The  learning  resources  Laura  used  for  these  projects  included  the  study  guides  in  the  WGU  learning  community, 
exemplary  sample  projects  done  by  previous  students,  articles  from  the  library,  and  review  of  textbooks  and 
Thomson  NetG  modules.  She  chose  a capstone  that  examined  the  human  resource  department  within  her  company 
in  light  of  the  research  and  academic  scholarship  about  successful  human  resource  operations.  Constant  guidance 
from  her  mentor  was  integral  to  the  successful  completion  of  these  items.  When  complete,  Laura  submitted  her 
materials  electronically  for  grading. 


Conclusion 

Since  students  utilize  learning  resources  primarily  to  refresh  and  develop  competencies  that  they  then  demonstrate 
through  assessment,  learning  resources  play  a unique  role  at  WGU.  Mentors  work  with  students  to  identify  to  best 
fit  of  LRs  that  will  enable  them  to  move  at  the  appropriate  pace  through  these  assessments  and  their  degree 
programs.  The  WGU  experience  with  learning  resources  demonstrates  how  a competency-based  higher  education 
institution  can  collaborate  with  other  educational  enterprises  to  provide  high  quality  instruction  that  fits  the  needs  of 
working  adult  students.  Such  innovation  leads  the  way  for  other  programs  to  work  on  similar  collaboratives  in 
system,  state,  national,  or  international  distance  education  consortia  or  initiatives. 
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Podcasting:  A New  Medium  for  Distance  Learning 


Christopher  Essex 
Indiana  University 

Abstract 

The  recent  popularity  of  podcasting  provides  the  distance  educator  with  a new  medium  for  delivering 
content  to  online  students.  Podcasts  are  audio  programs  delivered  to  users  via  the  Internet,  often  on  a subscription 
basis.  Instructors  can  present  via  podcast  much  like  they  would  to  a face-to-face  audience.  Students  can  access  the 
podcasts  while  they  are  living  their  busy  lives.  The  process  of  podcast  creation  is  described,  and  tips  for  successful 
podcasting  are  provided. 


Introduction 

Distance  education  has  too  long  been  tied  to  text  as  the  primary  means  of  delivering  instruction.  From  the 
old  correspondence  days  through  today,  the  vast  majority  of  information  communicated  to  the  online  student  has 
been  in  the  form  of  text,  whether  it  be  web  pages,  Acrobat  PDF  documents,  or  even  text-heavy  Flash  files.  As  an 
experienced  distance  educator,  having  taught  a fully  online  course  since  1998, 1 have  always  felt  that  this  was  a 
stifling  limitation.  Why  couldn’t  1 speak  to  my  distance  students  as  easily  as  I could  to  my  face-to-face  students? 
This  past  year,  I have  experienced  using  podcasts  as  the  way  to  break  through  this  wall  of  text. 

Definition  of  Podcast 

A podcast  is  a digital  audio  recording  that  can  be  downloaded  from  the  Internet.  Since  the  first  appearance 
of  the  term  online  in  February  2004  (Wikipedia,  2005),  podcasts  have  become  a hugely  popular  medium  for  news, 
information  and  entertainment.  It  has  been  estimated  that  4.8  million  people  downloaded  a podcast  in  2005  (Bridge 
Ratings,  2005).  Podcasts  are  designed  to  be  delivered  to  the  recipients  on  a regular  basis,  and  to  be  downloaded  to  a 
device  upon  which  they  are  played.  While  the  name  obviously  suggests  the  Apple  iPod,  an  external,  portable  USB 
or  Firewire  hard  drive  with  additional  hardware  and  software  to  play  digital  music  files,  podcasts  do  not  require 
iPods  for  delivery.  They  can  be  played  on  any  device  that  supports  MP3  files,  which  includes  any  modern  computer 
and  most  if  not  all  portable  digital  music  players.  140  million  such  devices  were  sold  worldwide  in  2005,  and  it  is 
projected  that  this  number  will  more  than  double  to  286  million  by  2010  (In-Stat,  2006).  Therefore,  access  to 
podcasts  should  not  be  a problem  for  most  distance  education  students. 

Another  important  aspect  to  podcasts  is  that  they  can  be  made  available  by  subscription.  The  instructor 
creates  a text  file  in  a relatively  easy-to-understand  XML  format,  called  an  RSS  file,  also  known  as  a feed.  RSS 
stands  for  Really  Simple  Syndication,  and  RSS  files  are  often  also  used  with  blogs  and  wikis,  as  well  as  news 
websites  such  as  CNN  (http://www.cnn.com).  The  RSS  file  is  composed  of  basic  information  about  the  podcast 
series,  such  as  title,  publication  date,  length,  a short  summary  of  the  content  of  each  episode,  and  most  importantly, 
the  web  addresses  for  each  episode.  The  RSS  file  is  uploaded  to  a web  server  along  with  the  podcast’s  MP3  file. 
Subscribing  to  a podcast  is  as  simple  as  entering  the  address  (URL)  for  the  series  into  a podcast  aggregator,  also 
known  as  a podcatcher.  Some  browsers  now  have  built-in  support  for  these  RSS  files.  Another  popular  way  to 
subscribe  to  podcasts  is  through  Apple’s  iTunes  program,  which  incorporates  a podcast  aggregator. 

Integrating  Podcasts  into  Distance  Education 

In  the  past,  I have  provided  my  online  students  with  a large  amount  of  reading,  including  the  course  text, 
selected  websites  and  scanned-in  articles  in  the  format  of  Acrobat  documents.  I would  also  provide  content  to  them 
through  discussion  forum  postings,  email  and  announcements  via  the  learning  management  system  (LMS).  I was 
dissatisfied  by  this,  however.  Often,  I would  find  myself  assigning  an  article  or  website  page  that  was  more-or-less 
what  I would  say  about  a subject,  but  not  exactly,  and  wishing  I could  directly  speak  to  the  students,  as  I would  if  we 
were  meeting  face-to-face. 

Podcasting  has  allowed  me  to  do  this.  Of  course,  I could  have  typed  out  an  essay  on  the  subject.  But 
speaking  is  faster  and  easier  for  me  than  typing,  even  with  today’s  voice  recognition  software.  Podcasting  allowed 
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me  to  present  content  to  my  distance  students  in  much  the  same  way  as  I would  if  they  were  sitting  in  front  of 
me  in  class. 


Podcast  Production 

For  recording  my  initial  podcasts,  1 used  the  built-in  microphone  and  speakers  in  my  Apple  eMac.  For 
software,  I used  iMovie  to  record  and  edit  the  audio,  and  iTunes  to  convert  it  to  an  MP3  file.  I did  this  because  I 
was  very  familiar  with  the  iMovie  program,  which  is  of  course  designed  for  use  with  video,  but  can  also  edit  and 
export  audio  tracks.  For  recent  episodes,  1 have  used  my  new  Apple  MacBook  Pro  laptop,  again  with  a built-in 
microphone  and  speakers.  For  software,  1 used  Apple’s  Garageband  3 (http://www.apple.com/ilife/garageband/), 
which  has  been  updated  to  include  features  specifically  for  podcasting.  For  interviews,  I have  used  both  Skype 
(http://www.skype.com),  in  combination  with  Audio  FliJack  to  record  the  session,  and  iChat  which  interfaces  with 
Garageband  directly.  But  podcasts  can  be  created  using  any  software  that  will  allow  you  to  record  audio  and  export 
it  as  an  MP3  file.  Many  Windows-  and  Linux-based  podcasters  use  the  Audacity  audio  editing  program 
(http://audacity.sourceforge.net/)  along  with  the  LAME  mp3  encoder 

(http://www.edhsonline.org/other/audacity/program/lame_enc.dll)  to  record  their  sessions.  If  your  computer  does  not 
have  a built-in  microphone,  or  if  you  want  to  upgrade  for  better  sound,  you  may  wish  to  purchase  an  external 
microphone  or  headset. 

Garageband  has  a couple  of  especially  useful  features  for  the  podcaster.  One  is  easy  access  to  other  sound 
files.  You  can  import  audio  files  from  your  iTunes  library  (though  of  course  copyright  issues  must  be  addressed)  and 
Garageband  also  includes  its  own  internal  database  of  sound  effects,  jingles  and  stingers  (small  snippets  of  music 
and/or  effects),  such  as  are  used  by  radio  disk  jockeys,  which  are  all  copyright- free  or  “podsafe.”  Other  sources  of 
“podsafe”  music  include:  the  Podsafe  Music  Network)  http://music.podshow.com/)  and  Podsafe  Audio 
(http://www.podsafeaudio.com/).  Another  advantage  of  Garageband  is  its  integration  with  iChat.  An  audio 
conference  in  iChat  can  be  recorded  in  Garageband,  with  each  side  of  the  conversation  separated  into  its  own 
channel,  for  ease  of  editing  and  stereo  effect. 

Usually  some  editing  of  the  audio  file  needs  to  be  made.  Coughing  fits,  cell  phone  rings,  pauses  for 
thought,  words  like  “um”  and  “well"  and  network  congestion-based  sound  glitches  all  need  to  be  taken  out,  in  order 
for  the  podcast  to  seem  professional.  An  answer  may  go  on  too  long,  or  the  subject  may  go  off  on  an  irrelevant 
tangent.  Sometimes  it  may  take  an  interview  awhile  to  really  get  started.  Even  if  you  are  simply  presenting  a 
lecture  recording,  you  should  edit  out  anything  that  is  not  meaningful  communication. 

Once  the  audio  file  was  in  MP3  format,  I uploaded  it  to  our  university’s  web  server,  and  created  a link  to 
the  file  from  the  relevant  page  on  my  website.  It  is  worth  pointing  out  that  podcasts  do  not  require  streaming 
software  to  be  installed  on  the  server,  since  they  are  meant  to  be  downloaded.  Since  I wanted  students  to  be  able  to 
subscribe  to  the  podcast  series,  I also  created  an  RSS  file  to  provide  aggregators  with  information  about  the  podcast. 

I did  this  by  hand,  but  there  are  programs  that  will  create  the  file  for  you,  such  as  Podifier 
(http://www.podifier.com/)  or  RSS  Publisher  (http://www.rsspublisher.com/).  At  my  university,  our  LMS  has 
recently  gained  an  integrated  podcasting  tool,  which  creates  the  RSS  file  automatically. 

If  you  wish  your  podcast  to  reach  an  audience  beyond  your  students,  you  may  want  to  publicize  it.  One 
way  is  to  enter  it  into  the  iTunes  podcast  database,  or  to  submit  it  to  a directory  site  with  an  Education  section,  such 
as  Podcast  Alley  (http://www.podcastalley.com/podcast_genres. php?pod_genre_id=7),  Yahoo!  Podcasts 
(http://podcasts.yahoo.com/search?t=l&p=education)  or  Educate  (http://recap.ltd.uk/podcasting/index.php). 

This  procedure  may  seem  somewhat  complicated  at  first,  but  once  you  have  produced  a couple  of  podcast 
episodes,  it  shouldn’t  be  that  time-consuming.  If  you  only  plan  to  podcast  once  a month  or  less,  you  may  want  to 
create  a brief  job  aid  for  yourself,  a paper  document  describing  the  basic  steps  of  the  podcasting  procedure  using 
your  hardware  and  software. 


Tips  for  Podcast  Creation  for  Distance  Education 

Here  are  some  specifics  I learned  during  my  first  semester  as  a podcaster: 

1 . Get  to  the  point.  Podcasts  should  be  small  packages  of  unified  content,  easy  to  take  in  during  a short 
walk  across  campus,  a session  at  the  gym,  or  a commute  to  work.  Chunk  your  material  into  thirty-minute-or-less 
podcasts.  Make  a Part  1 and  Part  2 if  absolutely  necessary  for  a specific  topic.  Remember,  student  time  is  as 
valuable  to  them  as  your  time  is  to  you. 

2.  Enjoy  yourself.  Just  as  with  an  in-class  lecture,  if  you  show  some  humor  and/or  personality  during  the 
podcast,  students  are  more  likely  to  continue  to  listen  to  you,  and  have  a better  attitude  and  higher  motivation  level 
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towards  the  course.  But  don’t  go  overboard;  don’t  get  silly.  As  the  Spinal  Tap  guys  say,  there’s  a fine  line  between 
clever  and  stupid.  Be  yourself— distance  students  often  complain  that  they  don’t  feel  like  they  know  their  instructor 
well — hearing  your  voice  can  help  them  get  a sense  of  you  as  a person. 

3.  Consider  having  a co-host  or  guest  speaker,  someone  to  offer  a different  aspect  on  an  issue.  Dialogue  is 
generally  more  interesting  to  listen  to  than  monologue.  Think  beyond  your  campus  borders  for  conversation 
partners;  with  the  widespread  availability  of  audioconferencing  and  high-speed  connections,  almost  anyone  in  the 
world  can  be  brought  into  your  distance  classrooms. 

4.  No  citations,  please.  Don’t  load  down  your  talk  with  references  and  URLs  for  the  students  to  check  out. 
Chances  are,  your  listeners  are  going  to  be  walking  across  campus,  in  a hallway  waiting  for  the  next  class,  at 
McDonalds  waiting  for  their  order,  or  in  similar  circumstances,  far  from  a pencil  and  paper  or  a computer.  Provide 
this  info  in  print  or  on  your  website  or  blog. 

5.  A small,  quiet  place.  While  Starbucks  may  be  where  you  do  your  best  thinking,  you  need  the  minimum 
of  background  noise  when  you  are  podcasting.  A computer  lab  may  not  be  much  better,  because  all  those  machines 
can  be  pretty  loud.  Even  the  fan  in  your  office  should  be  turned  off  when  you  are  recording. 

6.  Interviewing.  Should  you  rehearse  your  interview?  I would  suggest  going  over  the  questions  and 
possible  answers  with  your  subject  beforehand,  but  not  getting  into  a full  discussion.  This  way,  the  conversation 
will  sound  more  spontaneous,  and  both  of  you  will  be  more  interested  and  involved.  You  can  always  rerecord  an 
exchange  if  things  don’t  go  well,  but  it  isn’t  as  easy  to  get  back  enthusiasm  once  it’s  lost. 

7.  Encourage  feedback.  Invite  your  students  to  give  feedback  about  your  podcasts,  both  their  style  and 
content.  Did  you  speak  too  slowly,  too  fast?  Was  the  audio  quality  good  enough?  Was  the  amount  of  content 
presented  too  much  or  too  little?  What  needs  clarification?  You  may  want  to  create  an  online  survey  about  your 
podcasting,  in  order  to  measure  student  satisfaction.  After  all,  it  is  relatively  easy  to  go  back  and  edit  an  existing 
podcast,  and  to  replace  sections  that  are  not  effective. 

Integrating  Podcasts  into  Your  Course 

The  most  obvious  use  of  podcasts  is  to  record  lectures,  especially  if  you  have  previously  taught  the  course 
face-to-face  and  have  prepared  lecture  notes.  But  podcasts  can  also  be  used  to  provide  supplementary  material, 
extra  credit  content  or  material  relevant  to  only  a segment  of  your  students.  Podcasts  can  be  used  to  record  guest 
lecturers  or  one-time-only  presentations  or  discussions.  You  might  want  to  share  a conference  presentation  that 
you’ve  recently  given  with  your  students. 

There  is  other  audio  content  out  there  besides  your  voice  that  might  be  useful  to  your  students.  An  English 
professor  might  record  plays,  poems,  short  stories  or  essays  performed  by  others.  Field  interviews  are  an  important 
research  technique  in  many  fields;  these  can  be  presented  through  podcasts.  Historic  radio  shows  or  speeches  can  be 
distributed  (many  of  these  are  out  of  copyright).  Language  lessons  are  an  obvious  choice  for  this  new  medium;  why 
confine  your  students  to  a stuffy  language  lab  when  they  can  sit  in  the  park  and  repeat  their  vocabulary  words? 

Many  discipline  areas  have  audio  content:  students  can  learn  about  birdcalls  for  ornithology  class,  the  sounds  of 
instmments  in  music  history  class,  pronunciation  problems  for  speech  class. 

Another  use  for  audio  recording  is  to  give  student  feedback.  Instructors  do  enough  typing  in  a given  day; 
why  not  give  your  fingers  a break  and  provide  your  comments  on  student  work  in  audio  format?  In  some  cases,  this 
could  lead  to  longer  and  more  in-depth  commentary  from  an  instructor  who  is  weary  of  incessant  keyboarding.  (Of 
course,  this  type  of  audio  recording  may  not  precisely  fit  the  definition  of  podcasting.) 

It  is  also  important  to  consider  student-created  podcasts.  Your  online  distance  education  students  are  in 
front  of  a computer  anyway;  by  plugging  in  a headset  and  following  the  procedure  above,  they  could  be  making 
their  own  podcasts.  They  could  document  their  small  group  discussions  by  having  them  via  audioconferencing  and 
providing  a recording  for  the  instructor.  Individual  and  small-group  presentations  could  be  given  to  the  whole  class 
by  podcast,  and  then  shared  with  future  students  via  an  archived  file. 

Finally,  there  may  be  existing  podcast  series  that  might  be  relevant  for  your  students.  A quick  search  of  the 
Education  category  in  iTunes  shows  podcasts  on  grammar,  vocabulary,  math,  a large  number  of  foreign  languages, 
philosophy,  history,  biology. . .you  may  even  be  able  to  find  your  exact  course  content  as  presented  by  another 
professor. 


Student  Reaction  to  Podcasting 

I requested  student  comments  about  my  use  of  this  new  technology.  Here  are  some  representative 
comments  from  three  different  online  students: 
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“I  enjoyed  the  convenience  of  being  able  to  listen  to  a ‘lecture’  from  a place  other  than  a classroom.  The 
convenience  factor  is  the  biggest  benefit  1 see.” 

“I  downloaded  the  podcast  onto  [my]  memory  stick  and  took  it  [to]  listen  to  at  [my]  son’s  basketball 
practice!  I remember  having  to  get  someone’s  notes  when  you  missed  a lecture  - what  a pain!  Also  a good  way  to 
review  for  a test  by  listening  to  lecture  again!” 

“For  students  who  are  not  native  English  speakers,  having  a podcast  that  they  can  listen  to  and  stop  as  they 
need  to  grasp  the  meaning  [is]  a valuable  tool.” 

Overall,  student  reaction  to  this  instructional  innovation  was  positive.  The  fact  that  they  could  multitask — 
listen  to  course  content  while  accomplishing  other  tasks,  was  also  mentioned  by  a number  of  students.  Especially 
since  the  students  were  already  working  within  a mediated  environment,  the  transition  to  podcast  use  was  relatively 
easy.  Some  students  had  to  purchase  headphones  so  that  they  could  hear  the  podcasts  without  bothering  others,  so 
this  was  an  additional,  if  minimal,  expense. 


Conclusion 

Based  on  my  experience  in  a distance  education  setting,  I found  that  podcasts  can  provide  an  exciting  new 
option  for  online  instructors.  This  new  technology  can  provide  a fast,  efficient,  comfortable  way  to  deliver 
information  for  instructors,  and  for  distance  students,  it  provides  a convenient  way  to  access  and  review  information, 
as  well  as  to  present  it  themselves. 
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Predicting  Education  System  Outcomes:  A Scientific  Approach 

Theodore  Frick 
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Abstract 

Many  people  want  to  improve  education.  So  do  we.  Many  researchers  argue  that  such  change  must  be 
systemic,  since  attempts  at  piecemeal  change  have  made  little  difference  in  our  current  education  systems,  which 
still  operate  largely  as  they  have  in  the  past.  The  major  question  is:  What  should  we  change  in  the  current  education 
system,  and  how  do  we  know  it  will  work?  We  argue  that  what  we  need  is  a good  systems  theory  for  predicting 
outcomes  of  such  changes.  Otherwise,  we  may  be  making  changes  that,  while  well-intended,  may  not  result  in 
desired  outcomes — possibly  even  making  things  worse,  rather  than  better. 

In  this  paper  we  provide  an  overview  of  a change  strategy  that  is  inquiry-based  and  which  relies  on  a 
scientific  systems  theory  (AT1S).  This  is  followed  by  a simple  example  of  how  this  theory  can  be  applied  to  a 
fictional  school  system.  After  discussing  the  foundations  of  Axiomatic  Theories  of  Intentional  Systems  (ATIS)  and 
examples  of  systems  properties,  we  describe  computer  software  programs  now  under  development  that  allow  people 
to  apply  this  theory,  to  make  predictions,  and  to  measure  change:  1)  PESO:  Predicting  Education  System 
Outcomes,  and  2)  APT&C:  Analysis  of  Patterns  in  Time  and  Configuration.  APT&C  provides  an  alternative 
measurement  and  analysis  paradigm  for  investigating  dynamic  and  structural  relations  in  systems.  Finally,  we 
discuss  work  remaining  to  be  done  on  this  long-term  research  and  development  project  before  it  can  be  implemented 
in  practice. 

Overview 

Many  well-intentioned  people  want  to  improve  education.  So  do  we.  We  believe  that  education  could  be 
far  more  effective,  efficient  and  satisfying  than  it  is  in  our  current  educational  systems — both  K-12  and  higher 
education. 

Educators  who  have  taught  for  awhile  have  seen  several  widely  talked-about  changes  come  and  go.  For 
example,  some  of  the  innovations  have  been  referred  to  as:  site -based  management,  constructivist  classrooms, 
technology  integration,  school  restructuring,  and  yes,  even  systemic  change.  Educators  have  correctly  observed  that 
not  much  has  really  changed  from  what  they  can  see.  They  view  new  calls  for  change  with  a certain  detachment  and 
skepticism.  We  find  these  attitudes  understandable,  given  the  history  of  numerous  innovations  that  have  largely 
failed  to  make  significant  improvements  in  education.  Many  think:  “Just  another  buzzword.  Just  another  fad.  Ho- 
hum.” 

Why?  We  believe  that  the  following  questions  have  not  been  adequately  addressed: 

• “Change  what?” 

• “Change  how?”  and 

• “How  do  you  know  the  change  is  working?” 

We  must  know  what  to  change  in  order  to  know  how.  We  must  know  whether  the  change  accomplishes  the  goal  and 
that  the  change  does  not  have  negative,  unintended  effects.  Change  for  the  sake  of  change  is  nonproductive.  And, 
without  knowing  what  to  change,  the  “how”  is  irrelevant. 

As  an  analogy,  consider  an  old  bridge  that  is  failing — it  is  structurally  weak  and  is  impeding  the  flow  of 
traffic.  If  the  bridge  is  not  fixed,  it  will  collapse  and  vehicles  will  plunge  into  the  river.  When  engineers  design  a 
new  bridge,  they  utilize  adequate  scientific  theories.  No  one  in  modern  times  would  consider  designing  a new 
bridge  by  trial  and  error. 

Up  until  the  present,  we  have  had  no  valid  way  of  predicting  that  new  educational  system  designs  will  work 
any  better  than  what  we  now  have.  We  have  had  no  valid  way  of  describing  the  elements  of  any  educational  system 
or  of  evaluating  the  effects  of  change  throughout  the  system.  New  designs  and  curricula  have  been  patches — much 
like  fixing  rust  spots  on  an  old  car  with  body  filler  and  paint,  putting  on  new  seat  covers,  or  getting  new  tires.  The 
overall  structure  remains  unchanged. 

Many  researchers  have  focused  on  the  change  process.  We  believe  it  is  equally  important  to  focus  on  the 
outcomes  of  change — i.e.,  how  well  the  new  system  is  predicted  to  work  and  how  well  it  does  work.  We  need  both 
approaches — process  and  outcomes:  they  are  complementary.  The  change  process  could  be  effective,  but  the 
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resulting  new  system  may  not  have  the  desired  outcomes.  The  new  system  may  be  effective,  but  the  change  process 
may  leave  staff  and  families,  teachers  and  students  bitter  and  exhausted.  For  best  results,  both  processes  and 
outcomes  must  be  satisfactory. 

We  are  working  squarely  on  the  problem  of  predicting  education  system  outcomes.  The  predictions  must 
be  based  on  scientific  theory,  its  implications,  and  data  to  support  the  theory.  If  the  predictions  are  not  based  on 
scientific  theory,  then  how  can  we  justify  expending  great  effort  and  resources,  only  to  end  up  with  something  that  is 
no  better — or  possibly  even  worse — than  what  we  now  have?  It  is  no  wonder  that  educational  practitioners  often 
distrust,  resist  and  undermine  the  efforts  of  educational  reformers.  The  stakes  are  very  high.  The  consequences  of 
mistakes  can  be  devastating — particularly  when  changing  a whole  system  of  education. 

Understanding  systemic  change  is  not  a simple  matter.  Educators  will  need  to  leam  new  thinking  patterns. 
Hart  (1993)  has  noted  that  the  vast  majority  of  individual  belief  patterns  do  not  contain  dynamic  cycles.  Cognitive 
maps  of  belief  structures  tend  to  be  linear  with  few,  if  any,  feedback  loops.  Hart  indicated  that  exceptions  occurred 
with  those  people  in  professions  which  taught  them  to  think  in  dynamic  cycles  (e.g.,  ecologists,  systems  engineers). 
Similarly,  Senge  (2006)  has  provided  insight  into  business  organizations  by  identification  of  archetypal  patterns  of 
dynamic  cycles.  These  patterns  are  not  easily  described  or  understood  through  static  print  and  diagrams.  To  address 
this  problem  of  understanding,  Senge  and  his  colleagues  have  developed  role-playing  activities  and  computer 
simulations  in  order  to  help  business  people  understand  these  patterns  of  dynamic  relationships — some  of  which  run 
counter  to  individual  intuitions  about  how  systems  such  as  business  organizations  grow  and  change. 

For  these  reasons,  we  believe  that  it  will  be  very  helpful  to  educators,  if  they  can  use  computer  software 
that  will  help  them  to  design  new  educational  systems. 

SimEd  Technologies 

SimEd  Technologies  consist  of  four  parts: 

1.  The  ‘Get  Ready,  SET,  Go!’  change  model, 

2.  The  theory  model  options  set  called  Axiomatic  Theories  of  Intentional  Systems  (ATIS), 

3.  Computer  software:  Analysis  of  Patterns  in  Time  and  Configuration  ( APT&C ),  and 

4.  Computer  software:  Predicting  Education  System  Outcomes  (PESO). 

Designed  to  work  together,  SimEd  Technologies  use  computer  technology  to  help  describe  educational 
systems,  predict  system  changes  and  document  the  outcomes  of  change. 

We  will  describe  the  ‘Get Ready,  SET,  Go!’  model  to  predict  educational  system  outcomes  to  guide  the 
change  process.  This  inquiry-based  change  model  will  utilize  adequate  theory  and  computer  programs  which  are 
currently  under  development.  Then  we  will  give  more  detailed  discussion  of  other  parts  of  the  SimEd  Technologies. 
The  model  is  outlined  below. 

Get  Ready,  SET,  Go! 

Phase  1:  Get  Ready 

o Identify  the  specific  current  education  system  to  be  improved. 

o Over  some  interval  of  time,  measure  system  properties  using  our  computer  software  ATP&C 
(more  below). 

o Predict  outcomes  under  existing  conditions  if  nothing  is  changed  in  the  system  using  our  computer 
modeling  tool  PESO  (more  below). 

o If  these  outcomes  are  what  are  wanted,  then  do  not  modify  the  system.  However,  if  the  outcomes 
are  not  desired,  then  the  system  must  be  changed  so  that  the  desired  outcomes  can  be  obtained.  If 
change  is  desired,  proceed  to  Phase  2. 

2:  SET 

Use  PESO  software  to  model  newly  envisioned  educational  system  designs,  the  desired  feasible 
changes. 

Run  PESO  predictions  out  far  enough  in  time  to  make  sure  all  the  consequences  of  the  newly 
designed  system  would  be  acceptable.  This  iterative  process  will  determine  the  outcomes  of  the 
system  under  the  conditions  defined  by  the  changes.  Are  these  the  wanted  outcomes?  If  yes, 
proceed  to  Phase  3. 

• Phase  3:  Go! 

o Implement  the  new  design  chosen  in  Phase  2 in  the  education  system. 


• Phase 

o 

o 
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After  the  new  education  system  has  been  established,  then  over  some  interval  of  time,  measure 
system  properties  with  APT&C  software, 
o Verify  that  the  measures  confirm  the  predicted  system  outcomes.  If  not,  then  analyze  the  Phase-2 
processes  to  determine  what  modifications  are  required. 

PESO  Simulation 

We  are  building  a software  simulation  called  PESO:  Predicting  Education  System  Outcomes.  PESO  will 
model  system  concepts  and  allow  educators  to  focus  on  the  predictions.  PESO  is  a logic-based  simulation. 

The  most  familiar  simulations  are  scenario-based  programs  that  provide  “scripts”  to  determine  outcomes. 

A familiar  example  is  SimCity  (see  http://simcity.ea.com/  ).  Scripts  for  simulations  can  be  narrative  or  quantitative. 
Narrative  scripts  characterize  the  qualitative  parameters  of  a system — i.e.,  the  social,  philosophical,  and  individual 
descriptions  and  the  uncertainty  of  future  outcomes.  Quantitative  scripts  define  the  scientific  facts,  known  or 
credible  data,  and  quantitative  models  that  are  used  to  determine  future  outcomes.  However,  in  both  narrative  and 
quantitative  scripts,  the  content  is  closed.  There  are  a limited  number  of  possible  outcomes,  and  the  scripts 
predetermine  the  outcomes. 

If  the  script  lacks  fidelity,  then  users  may  learn  the  wrong  things.  For  example,  consider  what  might 
happen  if  modern  flight  simulators  that  are  used  to  train  military  and  commercial  pilots  lacked  fidelity.  A pilot  in 
the  simulator  might  discover  that  he  or  she  can  land  the  plane  successfully  in  a thunderstorm  with  lightning. 
However,  in  reality  such  an  action  would  be  dangerous  and  the  plane  could  crash.  These  would  be  devastating 
consequences  for  making  the  wrong  decision.  The  better  course  of  action  is  to  not  attempt  to  land  the  plane  under 
such  conditions,  and  wait  until  the  storm  passes.  Thus,  a simulation  script  that  lacks  fidelity  could  be  misleading 
and  dangerous. 

Friedman  (1999)  recognizes  these  kinds  of  problems  with  scenario-based  models  in  his  report,  “The 
Semiotics  of  SimCity,”  when  he  states: 

“Of  course,  however  much  ‘freedom’  computer  game  designers  grant  players,  any  simulation  will  be  rooted 
in  a set  of  baseline  assumptions.  SimCity  has  been  criticized  from  both  the  left  and  right  for  its  economic 
model.  It  assumes  that  low  taxes  will  encourage  growth  while  high  taxes  will  hasten  recessions.  It 
discourages  nuclear  power,  while  rewarding  investment  in  mass  transit.  And  most  fundamentally,  it  rests  on 
the  empiricist,  technophilic  fantasy  that  the  complex  dynamics  of  city  development  can  be  abstracted, 
quantified,  simulated,  and  micromanaged.”  (n.p.) 

On  the  other  hand,  logic-based  models  depend  on  the  logic  of  a theory  that  has  been  shown  to  be  valid  for 
the  targeted  empirical  system,  in  this  case,  an  education  system.  The  theory  describes  the  empirical  system  in  terms 
of  its  affect  relations,  properties,  and  axioms.  The  theory  is  then  used  to  project  outcomes  founded  on  the  theory 
with  respect  to  input  parameters.  The  instantiated  axioms  would  generate  a set  of  outcomes,  which  become  input 
parameters  that  instantiate  yet  more  axioms.  Unlike  scenario-based  models  that  are  closed  due  to  the  limited 
number  of  scripts,  logic-based  models  potentially  have  an  infinite  number  of  outcomes.  Such  models  are  more 
flexible. 

PESO  is  a logic-based  software  tool  that  makes  predictions  for  a specific  educational  system,  based  on 
current  conditions.  One  must  first  observe  properties  of  that  system  and  determine  how  the  values  of  those  system 
properties  change  over  some  time  period.  Properties  may  increase,  decrease,  remain  constant,  or  increase  to  some 
value  then  decrease.  When  those  changes  in  system  property  values  are  entered  into  PESO,  the  software  finds 
relevant  axioms  and  theorems  which  match  those  conditions,  and  then  executes  the  logic  of  Axiomatic  Theories  of 
Intentional  Systems  ( ATIS : Thompson,  2005).  PESO  effectively  applies  relevant  parts  of  the  theory  in  order  to 
make  predictions  of  what  will  happen  in  the  system. 

Significant  progress  has  been  made  on  PESO  software.  The  current  prototype  is  built  in  Flash  using  a 
programming  language  called  ActionScript.  Each  of  the  axioms,  antecedents,  consequents,  properties,  and  property 
attributes  are  treated  as  ‘objects’.  What  this  technical  capacity  of  software  means  is  that  the  software  can  be  easily 
extended  and  modified  as  the  theory  is  further  developed  and  validated.  In  effect,  PESO  handles  the  complexity  of 
the  theory  by  carrying  out  the  reasoning  according  to  the  theory  and  the  specific  conditions  that  are  typed  into  the 
software.  The  examples  and  figures  below  illustrate  how  PESO  does  the  reasoning  - based  on  the  axioms  and 
theorems  of  HITS'. 
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An  Example  of  PESO:  Predicting  Education  System  Outcomes 

In  the  United  States,  all  public  schools  are  affected  by  Ad  Child  Left  Behind  (2001)  legislation.  NCLB 
requires  schools  annually  to  assess  student  achievement  at  numerous  grade  levels.  Based  on  average  test  scores, 
schools  are  identified  as  succeeding  or  failing.  Schools  that  repeatedly  fail  to  meet  current  state  standards  for 
student  achievement  are  held  accountable.  Parents  have  the  opportunity  to  send  their  children  to  different  schools,  if 
their  present  school  is  not  succeeding. 

Consider  school  #9  in  Smithtown,  USA,  a fictitious  school  created  for  our  example.  Smithtown  #9  has  been 
identified  as  a failing  school.  If  a particular  school  is  identified  as  failing  according  to  state  standards,  NCLB 
permits  parents  to  move  their  children  to  a different  school.  What  would  happen  as  a consequence  of  falling 
enrollment?  Student  enrollments  are  a type  of  input  in  a school  system.  Axiom  13  predicts  that  decreasing  input 
implies  increasing  filtration.  Filtration  is  a system  property.  A filter  is  something  that  allows  certain  things  into  a 
system  but  not  others.  One  may  not  think  of  a label  of  “failure”  according  to  state  standards  as  a filter,  but  it  is. 


Figure  1.  ATIS  Axiom  13:  If  system  input  decreases,  then  filtration  increases. 


If  input  decreases 


Year  1 Year  2 Year  3 


Enrollment  falls 


In  this  example,  we  are  using  systems  language  that  is  not  familiar  to  most  educators.  In  each  graphic,  the 
system  property  (such  as  ‘filtration’)  and  its  value  (e.g.,  increases)  is  listed  for  an  educational  system.  Each  axiom  is 
an  “if  . . .,  then  ...”  statement  that  is  part  of  the  theory.  These  “if . . .,  then  ...”  statements  are  called  logical 
implications.  Axiom  13  states  that:  If  system  input  decreases,  then  filtration  increases.  This  is  not  a temporal 
relationship,  but  a logical  relationship.  If  it  is  true  that  input  decreases,  then  it  is  also  true  that  filtration  increases.  It 
does  not  matter  which  occurs  first. 

Does  the  systems  theory  make  any  other  predictions?  Yes,  PESO  identifies  axioms  11,10  and  16  as 
relevant.  See  Figures  2-4. 
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Figure  2.  Axiom  11:  If  system  input  decreases,  then  storeput  decreases. 


Figure  3.  Axiom  10:  If  system  input  decreases,  then  fromput  decreases. 


If  input  decreases 


Enrollment 


Then  fromput  decreases 


Hmm.. .there  aren’t  as  manj 
diplomas  to  print  this  year! 


Fewer  students  to  graduate 
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Figure  4.  Axiom  16:  If  system  input  decreases,  then  feedout  decreases. 


If  input  decreases 


Enrollment  falls 


The  predictions,  pictured  in  Figures  2-4,  tell  this  story.  If  enrollments  are  decreasing,  then  the  overall  number  of 
students  in  the  school  will  go  down,  and  eventually  fewer  will  be  eligible  to  graduate  and  leave  this  school. 

But  wait — there’s  more!  In  fact,  this  is  one  of  the  most  significant  features  of  the  PESO  simulation:  chains 
of  implications.  These  chains  are  based  on  the  premise:  If  A implies  B,  and  if  B implies  C,  then  A implies  C.  To 
continue  the  example,  Axiom  28  is  triggered  by  Axiom  13.  Axiom  28  states:  If  filtration  increases,  then 
adaptability  increases. 

How  could  Smithtown  School  #9  adapt?  Given  the  prediction  that  the  NCLB  label  of  ‘failing  school’  will 
result  in  a lower  student  enrollment,  actions  can  be  taken  to  prevent  that  from  occurring.  System  theory  embedded 
in  the  PESO  software  offers  Smithtown  School  options  for  actions  that  could  prevent  lower  enrollment.  Smithtown 
could  consider  actions  increasing  system  strongness  with  respect  to  instructional  affect  relations.  If  strongness  of 
instructional  affect  relations  is  increasing,  what  does  ATIS  predict? 

055:  If  strongness  increases,  then  hierarchical  order  decreases. 

056:  If  strongness  increases,  then  flexibility  increases. 

106:  If  strongness  increases,  then  toput  increases. 

107:  If  strongness  increases,  then  input  increases. 

108:  \f  strongness  increases,  then  fdtration  decreases. 

How  could  Smithtown  increase  strongness  of  instructional  affect  relations?  The  school  could  offer  more  guidance  of 
student  learning  by  bringing  in  teaching  aides,  either  paid  or  volunteer,  or  by  providing  more  instructional 
technology  that  can  actually  guide  learning.  Peer  tutoring  programs  in  which  more  advanced  students  coidd  tutor 
less  advanced  students  woidd  increase  the  guidance  of  learning.  As  can  be  seen  above,  the  theory  predicts  that  quite 
a few  things  would  change  in  the  system  if  strongness  were  increased. 

Axiom  56  states  that  if  strongness  increases,  then  flexibility  increases.  Flexibility  means  here  that  there  are 
more  different  kinds  of  alternative  paths  through  which  guidance  can  occur.  For  example,  the  teacher  could  be 
guided  himself  or  herself  by  e-learning  materials  and  then  guide  students,  and  likewise  for  teacher  aids.  Or  the 
teacher  can  instruct  some  students,  who  then  in  turn  instruct  others,  etc.  Filtration  is  decreased  by  removal  of  the 
“failure”  rating  by  meeting  NCLB  criteria  for  annual  yearly  progress.  Axiom  108  predicts  that  if  strongness 
increases,  then  filtration  decreases. 

There  are  additional  axioms  and  theorems  that  are  triggered  by  increasing  strongness  of  affect  relations,  but 
space  precludes  discussion  here. 


ATIS:  The  Basic  Concepts 

ATIS  (Axiomatic  Theories  of  Intentional  Systems)  is  a theory  model  options-set  that  is  designed  to 
construct  scientific  theory  for  certain  types  of  behavioral  systems.  In  particular,  it  is  used  to  develop  behavioral 
predictive  theories  and  technologies. 
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ATIS  is  founded  on  a basic  principle  that  we  all  rely  on  day-to-day  to  make  decisions  about  what  we  do. 

The  principle  is  based  on  Jerome  Bruner’s  (1990)  conclusion  about  how  we  derive  meaning  from  our  cultural 
contexts — i.e.,  the  systems  in  which  we  live: 

We  will  be  able  to  interpret  meanings  in  a principled  manner  only  in  the  degree  to  which  we  are 

able  to  specify  the  structure  and  coherence  of  the  larger  contexts  in  which  specific  meanings  are 

created  and  transmitted,  (pp.  64-65) 

We  normally  do  this  interpretation  and  integration  of  observed  phenomena  intuitively.  If  in  fact  our  world 
were  not  well-organized  and  intuitively  predictable,  we  would  not  be  able  to  function  in  our  daily  lives.  We  know 
that  if  we  show  up  for  work  and  do  what  we  are  supposed  to  do,  that  our  job  will  still  be  there  the  next  day — 
assuming  that  the  larger  context  in  which  we  work  does  not  change. 

Students  know  that  if  they  study  their  text  assignments,  listen  in  class  and  comprehend  what  the  instructor 
is  saying,  and  work  the  problems  for  the  class  in  such  a manner  that  they  get  the  correct  answers,  that  they  will 
receive  a grade  that  reflects  the  quality  of  their  work.  That  is,  if  a student  consistently  receives  an  “A”  on  quizzes, 
tests,  reports,  etc.,  then  that  student  expects  to  receive  an  “A”  for  the  course.  If  such  students  end  up  with  a “C”  for 
the  course,  they  know  that  there  is  “something  wrong.”  Why  is  there  something  wrong?  The  continual  integration  of 
data  into  their  thinking  gave  rise  to  a new  structure  that  reaffirmed  their  perceptions  that  they  were  doing  well. 

When  they  received  a “C”,  it  was  not  consistent  with  the  principles  upon  which  they  had  been  relying.  Their 
immediate  reaction — the  instructor  really  messed  up!  They  will  go  and  get  the  grade  changed  because  it  does  not 
reflect  the  structure  of  the  system  that  they  had  come  to  expect. 

ATIS  relies  on  the  observable  fact  that  our  lives  are  more  predictable  than  not. 

If  the  outcomes  are  not  what  we  expected,  then  we  did  not  have  full  knowledge.  The  behavioral  sciences  are  distinct 
from  the  physical  sciences  mainly  in  terms  of  what  we  actually  know  about  any  particular  event.  If  we  knew  more 
about  an  event,  our  prediction  may  have  been  correct.  We  all  believe  that  events  are  predictable — if  we  only  knew 
more.  It  is  the  basic  tenant  of  ATIS  that  such  is  correct. 

If  we  did  not  know  that,  if  we  treat  our  children  in  a certain  way,  they  will  respond  predictably,  then  child 
rearing  and  education  would  be  impossible.  The  slogan  that  “all  children  are  different”  is  a platitude,  but,  if  they 
were,  education  itself  would  not  be  possible.  Children  are  all  “different”  in  that  we  know  that  we  must  treat  them 
and  recognize  them  individually  if  we  want  them  to  achieve,  but  we  also  know  that  children  leam  by  just  such 
attention. 

Are  different  outcomes  predictable  for  educational  systems?  Of  course  they  are.  Differing  outcomes  can 
be  attributable  to  a variety  of  conditions  including  “attention  of  the  child,”  “teaching  skills  of  the  teacher,”  “intellect 
of  the  child,”  “intellect  of  the  teacher,”  and  “physical  surroundings”.  The  question  is  not  whether  an  event  is 
predictable,  but  whether  we  know  what  we  need  to  know  in  order  to  make  the  prediction.  ATIS  helps  to  focus 
attention  on  what  one  needs  to  know,  and  provides  the  structure  to  make  a reasoned  decision  concerning  the 
outcomes. 

Background  of  ATIS:  General  Systems  Theory 

The  concept  of  general  systems  theory  (GST)  was  first  introduced  by  Ludwig  von  Bertalanffy  in  1949. 
Bertalanffy  (1968)  argued  that  there  exists  a general  theoiy  that  could  characterize  the  behavior  of  systems, 
regardless  of  whether  these  are  scientific,  natural  or  social;  and  he  proposed  GST  as  an  interdisciplinary  theory  that 
could  contribute  to  the  unity  of  science.  System  behavior  results  from  the  relationships  between  its  components,  and 
is  not  just  a simple  summation  of  its  parts.  The  characteristics  of  each  system  component  therefore  cannot 
adequately  explain  how  the  system  itself  behaves. 

Since  then,  there  have  been  extensive  contributions  by  others  in  the  development  of  GST  as  a logical  and 
mathematical  theory  to  provide  an  “exact  language  permitting  rigorous  deductions  and  confirmation  (or  refusal)  of 
theory”  (Bertalanffy,  1972,  p.30).  Others  have  also  contributed  well-developed  descriptive  theories  (e.g.,  Wymore, 
1967;  Cornacchio,  1972;  Mesarovic  & Takahara,  1975;  Lin,  1987;  Lin,  1999;  Bar-Yam,  2003).  In  education,  GST 
has  been  used  by  researchers  to  discuss  educational  systems  design  and  systemic  change,  but  these  approaches  have 
not  been  grounded  in  scientific  theory  about  educational  systems  (Banathy,  1991;  Caine  & Caine,  1997;  Duffy, 
Rogerson  & Blick,  2000;  Senge,  Cambron-McCabe,  Lucas,  Smith,  Dutton  & Kleiner,  2000).  Rather  these 
approaches  largely  describe  processes  through  which  organizations  can  change,  not  whether  those  changes  are  likely 
to  result  in  desired  outcomes. 

The  SIGGS  theory  model  provided  the  first  extensive  formalization  of  a GST  model  for  educational 
theorizing  (Maccia  & Maccia,  1966;  Steiner,  1988).  Through  the  synthesis  of  four  theories:  Set,  Information,  di- 
Graph,  and  General  Systems,  SIGGS  provided  a logical  description  of  general  system  properties,  which  enabled 
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retroduction  of  201  hypotheses  in  a theory  of  school  systems.  Flick,  Hood,  Kirsch,  Reigeluth,  Walcott  and  Farris 
(1994)  extended  Maccia  and  Maccia’s  work  by  classifying  the  system  properties  into  basic,  structural,  and  dynamic 
properties.  This  classification  recognized  that  some  SIGGS  properties  were  structural  as  they  described  the 
connectedness  between  system  components  ( SIGGS  Website,  1996a).  Yet,  others  were  dynamic  and  described  how 
patterns  of  relationships  between  system  components  are  altered  due  to  changes  within  the  system  or  between  the 
system  and  its  environment  ( SIGGS  Website,  1996b).  Thompson  (2005)  recognized  that  the  structural  properties 
essentially  defined  the  system  topology. 

To  provide  a theory  that  is  logically  and  mathematically  sound,  a system-descriptive  axiom  set  is  needed. 
Although  SIGGS  was  fairly  comprehensive,  there  was  no  attempt  to  analyze  the  201  hypotheses  for  consistency  nor 
to  finalize  an  axiom  set  that  would  be  the  underlying  axioms  for  a GST.  Thompson  has  since  been  developing 
Axiomatic  Theories  of  Intentional  Systems  ( ATIS ),  which  is  a logico-mathematical  theory  model  for  analyzing  and 
predicting  behavior  of  systems  that  are  goal-directed  or  intentional.  Using  the  original  SIGGS  hypotheses, 
Thompson  developed  a nomenclature  to  define  system  properties,  which  improved  the  precision  with  which  SIGGS 
properties  could  be  used  (Thompson,  2005).  Thompson  also  identified  an  initial  list  of  approximately  100  axioms 
(subject  to  change,  as  this  work  is  on-going),  and  extended  the  73  SIGGS  general  system  properties  to  136  in  ATIS 
{APT&C  Website,  2005). 

This  is  Getting  Pretty  Technical — How  can  It  Be  Managed? 

The  busy  education  professional  may  wonder,  “Do  I have  to  be  a mathematician  to  benefit  from  these 
concepts  in  my  work?”  The  short  answer  is:  No,  you  don’t  have  to.  We  don’t  have  to  be  engineers  to  drive  our  cars 
or  use  our  microwave  ovens.  We  can  use  devices  built  on  scientific  theories  without  knowing  all  the  details. 

ATIS  is  quite  complex  and  very  detailed.  It  is  difficult,  even  for  the  present  authors,  to  keep  track  of  all  the 
detail.  This  is  where  we  believe  that  computer  technology  can  help  us.  We  are  building  a software  simulation, 
called  PESO'.  Predicting  Education  System  Outcomes.  PESO  will  keep  track  of  all  the  details,  allowing  us  to  focus 
on  the  predictions. 

How  PESO  Makes  Logic-based  Predictions 

Even  though  there  are  over  200  axioms  and  theorems  in  ATIS  as  of  this  writing  (APT&C  Website,  2005), 
only  5 axioms  apply  under  the  condition:  input  decreases.  Axioms  10,  1 1 and  13  predict  thee  outcomes  of 
decreasing  input.  However,  Axiom  1 1 predicts  a decrease  in  storeput,  which  triggers  Axiom  16.  Similarly,  Axiom 
13  triggers  Axiom  28.  This  kind  of  chaining  illustrates  how  the  inference  engine  that  is  built  into  PESO  works. 

PESO  actualizes  the  logical  implication  of  transitivity  - e.g.,  if  A implies  B,  and  if  B implies  C,  then  A implies  C. 

PESO  will  carry  out  the  implications,  as  illustrated  in  Figures  1-4  above.  First,  the  user  must  enter  the 
specific  conditions  that  currently  exist  for  a particular  school  system  or  district.  PESO  then  finds  all  the  relevant 
axioms  and  theorems  from  A TIS  and  uses  them  to  make  predictions  about  this  particular  system — not  other  systems, 
not  all  systems,  but  this  system  under  these  conditions. 

How  will  you  know,  for  example,  whether  input  is  increasing,  flexibility  is  decreasing,  or  filtration  is 
increasing  in  your  particular  education  system?  You  will  need  to  measure  these  system  properties.  This  means  you 
will  need  to  observe,  collect  data,  and/or  use  existing  data  about  your  education  system.  You  will  be  able  to  use 
APT&C  software  to  assist  in  the  data  collection  and  analysis.  This  will  help  you  identify  the  temporal  and  structural 
patterns  in  your  education  system,  and  it  will  do  the  calculations  for  you. 

SimEd  Technologies  Will  Include  APT&C  Software 

Analysis  of  Patterns  in  Time  and  Configuration,  APT&C,  is  a different  kind  of  measurement  paradigm. 
APT&C  is  a mixed-mode  research  methodology  and  software  tool  to  help  create  knowledge  of  education  systems 
that  is  directly  linked  to  practices  and  changes  in  practices.  APT&C  bridges  the  gap  between  traditional  linear 
models  in  quantitative  research  and  qualitative  research  findings  that  lack  generalizability  (Frick,  1990;  2005). 
APT&C  builds  on  work  done  by  Frick  (1990)  on  APT  and  by  Thompson  (2005). 


147 


APT&C  is  different  from  the  widely  used  Statistical  Package  for  the  Social  Sciences.  SPSS  uses  the 
traditional  approach  to  measurement  and  statistics  that  requires  you  to  measure  things  separately,  and  enter  numbers 
for  each  variable  such  as  a student’s  test  score,  age  or  grade  in  school.  Then  you  analyze  the  data  by  using  statistics 
such  as  correlation,  analysis  of  variance,  regression  analysis,  etc.  This  is  referred  to  as  a linear  models  approach  by 
statisticians.  Linear  models  statistically  relate  separate  measures  of  things. 

In  contrast,  APT&C  directly  measures  the  relation.  The  difference  is  significant.  In  the  linear  models 
approach,  you  will  get  an  r value  or  the  results  of  an  F test,  for  example,  to  tell  you  whether  a relation  between  or  among 
measures  is  statistically  significant.  In  APT&C  you  will  get  different  kinds  of  values  which  are  measures  of  temporal  or 
structural  patterns.  For  example,  you  could  predict  student  engagement  when  direct  instruction  is  or  is  not  occurring  as 
Frick  (1990)  did.  He  found  that  students  were  13  times  more  likely  to  be  off-task  when  direct  instruction  was  not 
occurring  during  academic  activities.  This  is  a temporal  pattern.  In  his  study,  students  were  observed  to  be  engaged 
about  97  percent  of  the  time  during  direct  instruction,  but  only  57  percent  of  the  time  during  non-direct  instruction. 

These  percentages  are  measures  of  the  temporal  relation,  and  are  based  on  probability  theory  and  set  theory. 

This  kind  of  APT&C  finding  is  similar  to  epidemiological  findings  in  medicine.  For  example,  heavy  cigarette 
smokers  are  5-10  times  more  likely  to  have  lung  cancer  later  in  their  lives  (Kumar,  et  ah,  2005),  and,  if  they  quit 
smoking,  the  likelihood  decreases.  While  causal  conclusions  cannot  be  made  in  the  absence  of  controlled  experiments, 
nonetheless  one  can  make  practical  decisions  based  on  such  epidemiological  evidence.  You  can  do  likewise  with 
APT&C.  The  practical  conclusion  of  Frick’s  study  is  that  direct  instruction  engages  students.  If  a teacher  wants  students 
to  learn,  direct  instruction  is  more  likely  to  produce  student  engagement. 

In  addition  to  temporal  properties,  APT&C  will  allow  you  to  measure  structural  properties  of  educational 
systems.  Examples  of  structural  properties  were  listed  in  Figures  1-4  above,  such  as  strongness  and  flexibility.  You 
will  enter  data  into  what  are  called  ‘affect  relation  matrices’  to  indicate  the  structure  or  configuration  of  your 
educational  system.  Then  the  software  will  “crunch  the  numbers”  and  provide  the  values  for  properties  such  as 
strongness  and  flexibility.  This  is  how  you  will  determine  whether  strongness  or  flexibility  is  increasing  or 
decreasing  over  some  period  of  time. 

Once  you  have  measured  and  analyzed  these  dynamic  and  structural  patterns  in  your  education  system, 
then  you  can  identify  the  specific  conditions  that  exist  regarding  those  property  values  of  your  educational  system. 

You  use  the  PESO  program  to  then  make  predictions  of  educational  outcomes  for  your  system  under  these  specific 
conditions.  If,  for  example,  strongness  of  instructional  affect  relations  is  decreasing  in  your  system,  PESO  will 
apply  different  axioms  than  if  it  is  increasing. 

Further  information  on  APT&C  and  additional  references  are  found  in: 
http://education.indiana.edu/~aptfrick/. 


Next  Steps 

SimEd  Technologies  are  theories,  methodologies  and  software  tools  to  describe  complexity  in  educational 
systems.  PESO  will  need  to  be  tried  out  and  validated  in  real  educational  systems,  whether  schools  or  school 
districts,  charter  schools,  alternative  schools,  or  school  to  work  programs.  When  it  is  established  that  PESO 
adequately  describes  and  predicts  educational  system  outcomes,  educators  can  use  SimEd  Technologies  to  model  the 
consequences  of  educational  systems  changes.  SimEd  Technologies  will  show  educators  all  the  consequences,  even 
the  unintended  consequences,  of  changing  one  part  of  the  complex  educational  systems  they  direct.  Better  changes 
and  better  predictions  of  outcomes  will  result. 

For  more  current  information  on  the  SimEd  project,  please  visit  the  Website  at: 
http://education.indiana.edu/~simed/ . 


References 

APT&C  (2005).  Research  reports.  Retrieved  April  10,  2006,  from  http://www.indiana.edu/~aptfrick/reports/. 
Banathy,  B.  (1991).  Systems  design  of  education.  Engelwood  Cliffs,  NJ:  Educational  Technology  Publications. 
Bar-Yam,  Y.  (2003).  Dynamics  of  complex  systems.  Boulder,  CO:  Westview  Press. 

Bertalanffy,  L.  von,  (1968).  General  system  theory:  Foundations,  Development,  Applications.  New  York:  George 
Braziller. 

Bertalanffy,  L.  von,  (1972).  The  History  and  Status  of  General  Systems  Theory.  In  G.J.  Klir  (Ed.):  Trends  in 
general  systems  theory.  New  York:  Wiley-Interscience. 

Bruner,  J.  (1990).  Acts  of  meaning.  Cambridge,  MA:  Harvard  University  Press. 


148 


Caine,  R.  & Caine,  G.  (1997).  Education  on  the  edge  of  possibility.  Alexandria,  VA:  Association  for  Curriculum 
Supervision  and  Development. 

Cornacchio,  J.  V.  (1972).  Topological  concepts  in  the  mathematical  theory  of  general  systems.  In  G.  J.  Klir  (Ed.), 
Trends  in  general  systems  theory.  New  York:  Wiley-Interscience,  303-339. 

Duffy,  F.,  Rogerson,  L.  & Blick,  C.  (2000).  Redesigning  America’s  schools:  A systems  approach  to  improvement. 
Norwood,  MA:  Christopher-Gordon  Publishers. 

Friedman,  T.  (1999).  The  Semiotics  of  SimCity.  First  Monday,  4(4).  Retrieved  April  10,  2006,  from 
http://www.firstmonday.dk/issues/issue4  4/friedman/ . 

Frick,  T.  (2005).  Bridging  qualitative  and  quantitative  methods  in  educational  research:  Analysis  of  Patterns  in 
Time  and  Configuration  (APT&C).  Retrieved  April  10,  2006  from 
http://education.indiana.edu/~frick/proposals/apt&c.pdf. 

Frick,  T.  (1990).  Analysis  of  Patterns  in  Time  (APT):  A Method  of  Recording  and  Quantifying  Temporal  Relations 
in  Education.  American  Educational  Research  Journal,  27(1),  180-204. 

Frick,  T.  (1991).  Restructuring  education  through  technology’.  Bloomington,  IN:  Phi  Delta  Kappa  Education  Foundation. 

Frick,  T.  W.,  Flood,  P.  Kirsch  K.,  Reigeluth  C.,  Walcott  A.,  and  Farris  H.  (1994),  Simulosophy  Group  Report:  Sixth 
International  Conference  on  the  Design  of  Social  Systems.  Retrieved  April  10,  2006  from 
http://www.indiana.edu/~tedfrick/simulosophy.pdf. 

Flart,  J.  (October,  1993).  Cognitive  maps.  Presentation  at  the  Cognitive  Science  Colloquium  Series,  Indiana 
University,  Bloomington. 

Jenlink,  P.,  Reigeluth,  C.,  Carr,  A.  & Nelson,  L.  (1996  - present,  in  process).  Facilitating  systemic  change  in  school 
districts:  A guidebook.  Unpublished  manuscript. 

Lin,  Y.  (1987).  A model  of  general  systems.  Mathematical  modeling,  9(2),  95-104. 

Lin,  Y.  (1999).  General  systems  theory:  A mathematical  approach.  NY:  Kluwer  Academic/Plenum. 

Maccia,  E.S.  & Maccia,  G.S.  (1966).  Development  of  educational  theory  derived  from  three  theoiy  models.  Washington, 
DC:  U.S.  Office  of  Education,  Project  No.  5-0638. 

Mesarovic,  M.  D.  & Takahara,  Y.  (1975).  General  systems  theory:  Mathematical  foundations.  In  R.  Bellman  (Ed.), 
Mathematics  in  science  and  engineering,  Vol.  113.  NY : Academic  Press. 

National  Commission  on  Excellence  in  Education  (1983).  A nation  at  risk.  Washington,  DC:  U.S.  Governmental  Printing 
Office. 

Reigeluth,  C.  (1992).  The  imperative  for  systemic  change.  Educational  Technology’,  32(11),  9-12. 

Senge,  P.  (2006).  The  fifth  discipline:  The  art  and  practice  of  the  learning  organization.  NY : Currency/Doubleday. 

Senge,  P.,  Cambron-McCabe,  N.,  Lucas,  T.,  Smith,  B.  Dutton,  J.  & Kleiner,  A.  (2000).  Schools  that  learn.  New  York: 
Doub  leday/  Currency . 

SIGGS  Website  (1996a).  SIGGS  Structural  Properties.  Retrieved  April  10,  2006  from 
http://www.indiana.edu/~tedfrick/siggs3.html . 

SIGGS  Website  (1996b).  SIGGS  Dynamic  Properties  (temporal  change).  Retrieved  April  10,  2006  from 
http://www.indiana.edu/~tedfrick/siggs4.html . 

Steiner,  E.  (1988).  Methodology’  of  theory’  building.  Sydney:  Educology  Research  Associates. 

Thompson,  K.  R.  (2005).  “General  System”  defined  for  predictive  technologies  of  A-GSBT  (Axiomatic-General 
Systems  Behavioral  Theory).  Manuscript  accepted  for  publication,  Scientific  Inquiry  Journal. 

Wymore,  A.  W.,  (1967),  A Mathematical  Theory  of  Systems  Engineering:  The  Elements.  New  York:  Wiley. 


149 


From  Design  Theory  to  Development  Practice : Developing  a Stronger 

Understanding  of  Our  Field. 


Aaron  Fried 
Yongjin  Lee 
Karen  Zannini 
Tiffany  A.  Koszalka 
Syracuse  University 


Abstract 

How  does  one  develop  deep  knowledge  of  our  field?  Inherent  in  this  quest  is  a complex 
field  with  multiple  definitions  and  research  perspectives  on  our  practices.  This  session 
will  showcase  an  approach  that  engages  students  in  generative  activities  to  prompt  critical 
thinking  about  design  and  development  theory  and  practices.  Resulting  instructional 
design  theory  databases  collaboratively  created  by  classmates  and  instructional 
development  models  with  accompanying  detailed  work  plans  created  by  individuals  will 
be  showcased  and  discussed. 


Introduction 

How  does  one  develop  deep  knowledge  of  our  field?  Inherent  in  this  quest  is  a complex 
field  with  multiple  definitions  and  research  perspectives  on  our  practices.  This  paper 
showcases  an  approach  that  engages  students  in  generative  activities  to  prompt  critical 
thinking  about  design  and  development  theory  and  practices. 

To  become  competent  practitioners,  those  new  to  the  instructional  design  and 
development  (IDD)  community  must  acquire  advanced  knowledge  and  skill 
competencies  to  develop  and  research  new  theories  and  continually  enhance  the  state  of 
our  profession.  Lave  and  Wenger  (1991)  suggest  that  newcomers  are  acculturated 
through  processes  beginning  with  legitimate  peripheral  participation,  or  participating  on 
the  side  line  through  a process  of  observational  learning  and  limited  participation  that 
eventually  leads  into  in-depth  engagement  with  our  core  practices.  Through  processes  of 
production  (e.g.,  training,  feedback,  involvement,  leadership,  etc.)  newcomers  eventually 
move  into  a role  of  full  participation  as  they  become  recognized  as  members  of  a 
community. 

In  this  paper  we  describe  and  showcase  two  activities  and  the  resulting 
deliverables  designed  to  engage  IDD  doctoral  students  in  generative  activities  (Wittrock, 
1990)  to  expand  depth  of  knowledge  of  instructional  design  theory  and  development 
practices.  As  newcomers  it  is  important  that  each  individual  be  able  to  discern  for 
him  /herself  what  our  field  is  about,  how  the  community  members  have  defined, 
practiced,  and  researched  our  field,  and  how  each  individual  will  in  turn  contribute  to  our 
profession.  Instructional  activities  facilitating  newcomers  in  building  understanding  and 
communicating  new  knowledge  are  important  to  growth  in  the  community. 
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What  is  Instructional  Design  and  Development? 

What  should  newcomers  ‘know’  about  IDD  practices?  Members  of  a community  share 
definitions  and  ideals  about  how  they  will  practice  (Lave&  Wenger,  1991).  Practitioners 
have  developed  a multiple  paradigms  and  approaches  for  designing,  developing, 
implementing,  and  evaluating  instruction,  each  based  on  different  understandings  of 
learning  and  instruction,  and  arguably,  each  having  its  advantages  and  challenges.  This 
complexity  of  theories  and  practices  can  be  challenging  for  newcomers.  How 
‘instructional  design’  and  ‘instructional  development’  are  defined  impacts  our  actual 
practice.  It  is  essential  to  characterize  similarities  and  differences  among  the  multitude  of 
IDD  paradigms  and  unpack  the  complexity  of  each  to  perform  well  within  the  community 
and  help  to  enhance  practices. 

Reigeluth  (1999)  suggests  that  instructional  design  theories  have  four  basic 
characteristics.  Design  theories  (i)  prescribe  how  instruction  should  be  organized,  (ii) 
include  instructional  methods  and  situations  where  there  use  is  acceptable,  (iii)  have 
methods  that  can  be  broken  down  in  to  more  detail,  and  (iv)  increase  the  probability  that 
instructional  outcomes  will  be  achieved.  Instructional  design  theories  provide  a 
framework  from  which  instruction  is  organized  and  delivered  to  prompt  learning. 

Instructional  development  is  often  described  as  a systematic  process  for  producing 
instruction.  Gustafson  and  Branch  (1997)  have  described  several  instructional 
development  models  accounting  for  a variety  of  perspectives.  These  models  possess 
common  steps  yet  have  professed  linear,  iterative,  or  rapid  prototyping  approaches:  (i) 
objective-based  or  open-ended  in  terms  of  outcomes;  and  (ii)  specific  adaptation  for 
teacher,  business  practitioner,  evaluators,  or  for  specific  delivery  mediums  such  as 
classroom,  self-study,  computer-based,  or  distributed  learning  environments. 

Instructional  development  models  thus  provide  a framework  to  identify,  organize, 
produce,  deliver,  and  evaluate  instruction. 

There  exists  contradictory  usage  of  the  terms  instructional  design  and 
instructional  development  within  the  literature.  Recently  Gustafson’s  and  Branch’s 
(1997)  work  on  development  models  was  published  under  the  title  of  “Instructional 
Development,”  yet  some  of  Branch’s  earlier  work  describes  the  same  ideas  using  the  tenn 
“Instructional  Design  Models”  (Edmonds,  Branch,  & Mukherjee,  1994).  This 
contradictory  usage  of  the  terms  can  be  challenging. 

Educating  Our  Community  Members  To  Advance  Practices 

The  complexity  and  discrepancies  within  the  field  form  important  foundations  from 
which  practices  have  emerged  and  the  roots  upon  which  our  thinking  about  how  our  field 
should  grow  and  change  in  the  future.  In  this  doctoral  seminar,  learners  are  prompted  to 
think  critically  about,  explore,  and  then  describe  the  themes  among  our  IDD  practice 
literature.  A generative  and  social  learning  perspective  was  chosen  to  engage  learners  in 
this  thinking  and  working  so  that  each  member  of  the  class  contributed  his  or  her  own 
knowledge  into  a shared  understanding. 
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Wittrock  (1990)  suggests  that  learning  requires  active  participation  of  the  mind. 
Comprehension  occurs  by  formulating  connections  between  perceived  information,  prior 
knowledge  and  other  memory  components.  Learners  build  deeper  knowledge  by 
physically  and  mentally  manipulating  models  and  information  while  actively  seeking  to 
organize  and  integrate  informational  relationships  between  what  is  seen,  heard,  felt,  read, 
and  mentally  processed.  Further,  engaging  in  activities  collaboratively  exposed  learners 
to  other  member’s  interpretations  to  help  each  better  understand  the  meaning  of  ideas  and 
concepts  within  the  varied  IDD  field. 

Major  Course  Projects 

The  two  major  activities  accompanied  by  readings  and  discussions  within  the  course 
include:  social-generative  class  project  and  generative  practice  individual  project. 

Social-Generative  Class  Project 

Jonassen,  Carr,  and  Yueh  (1998)  have  described  databases  as  mind  tools  that  help 
breakdown  information  into  smaller  categories  while  allowing  for  interpretation  of 
information.  Learners  in  this  seminar  contribute  to  the  creation  of  an  online  database  of 
design  theories.  Learners  define  instructional  design  theories  and  create  viable  categories 
for  comparison  among  the  theories.  Secondly,  learners  identify  and  read  about  multiple 
theories  to  decide  if  they  ‘fit’  within  the  group’s  established  definitions  and  if  so, 
populate  the  data  base  with  pertinent,  agreed  upon  information. 

Generative  Practice  Project 

Learners  create  their  own  development  models  and  associated  work  plans.  Each  includes 
a visual  representation,  a brief  description  of  how  the  model  works,  and  a detailed  work 
plan  including  a schedule  of  all  tasks.  The  model  is  robust  so  that  any  type  of  instruction 
could  be  developed  using  this  model. 

Each  deliverable  is  showcased  with  explanations  of  its  fonnation,  how  the 
literature  informs  ideas,  and  strengths  and  weaknesses.  The  audience  is  prompted  to 
provide  critique,  ask  questions,  share  personal  experiences,  and  contribute  to  the 
discussion.  The  following  questions  provide  a beginning  framework  for  the  discussion: 

1 . How  do  we  understand  the  similarities  and  differences  between  the  terms 
instructional  design  and  instructional  development? 

2.  How  does  developing  a database  of  instructional  design  theories  help  us  develop 
depth  of  knowledge?  What  challenges  did  we  encounter?  How  would  we  change 
the  database  now? 

3.  How  does  creating  an  instructional  development  model  and  accompanying  work 
plan  help  us  develop  depth  of  knowledge?  What  challenges  did  we  encounter? 
How  would  we  change  the  activity  now? 
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Reflection  to  the  questions 


Social-Generative  Class  Project:  Building  a Design  Database 
What  is  an  instructional  design  theory?  How  is  a design  theory  different  than  an 
instructional  development  model?  What  makes  a theory  an  instructional  design  theory? 
These  are  the  questions  that  the  class  started  to  try  and  describe. 

Jonassen,  Carr,  and  Yueh  (1998)  have  described  the  benefits  of  constructing 
databases  in  learning  and  organizing  information.  In  order  for  a learner  to  be  able  to 
construct  a database,  the  learner  has  to  be  able  to  i)  structure  information  about  their 
topic,  ii)  identify  and  create  appropriate  and  relevant  fields,  and  iii)  construct  the  database 
based  on  potential  queries  that  would  be  helpful  and  relevant.  In  order  to  answer  the  main 
questions  for  this  project,  the  five  students  in  the  class  worked  to  work  through  these 
exact  problems,  i)  What  is  the  definition  of  an  instructional  design  theory;  how  can  in 
instructional  design  theory  be  identified?  ii)  What  are  the  important  and  relevant  fields 
that  should  be  used  in  creating  the  database?  iii)  What  are  the  potential  queries  that  could 
be  employed  with  this  database  and  how  should  that  drive  the  organization  and 
development  of  the  database?  In  addition  to  the  design  of  the  database,  students  had  to 
identify  and  index  a total  of  50  instructional  design  theories  in  the  database. 

This  process  provided  the  ultimate  experience  for  learning  about  the  instructional 
design  models.  A working  representation  of  the  finished  product  can  be  seen  at 
http://web. cortland.edu/frieda/ID/IDdatabase.html.  While  this  represents  the  finished 
product  in  terms  of  completing  the  course,  this  by  no  means  represents  a finished  product. 
The  initial  phases  of  the  project  were  spent  defining  and  describing  design  theories.  Much 
of  this  work  was  based  on  the  two  volumes  edited  by  Charles  Reigeluth  (1983,  1999). 

The  students  created  a working  definition  of  instructional  design  theories  such  that  an 
instructional  design  theory:  (i)  is  prescriptive,  (ii)  describes  its  acceptable  uses,  (iii)  has 
detailed  methods,  and  (iv)  increases  the  probability  that  learning  will  occur  (see  also  the 
more  detailed  definition  above). 
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Instructional  Design  Theory  Database  Project 

Syracuse  University  School  of  Education  Department  of  Instructional  Design.  Development.  & Evaluation 


Fields  contained  in  the  database  records: 

Theory  Name 
Authors 

Associated  Learning  Theory 
Model  Description 

Specification  of  Theory 
(a)  Goals  and  preconditions 
(b)  Principles 
(c)  Condition  of  learning 

(d)  Required  media 

(e)  Role  of  facilitator 
(f)  Instructional  strategies 

(g)  Assessment  method 

Formative  Research  & Application 
(a)  Tested  context 
(b)  Research  method 
(c)  Research  description 
(d)  Resources 

For  definitions  of  each  of  these  field  headings,  click  here. 

ran 

Search  ID  Thoeries 

v [ Search  ] 

Find  ANY  word  v] 


Sita  Seath  bv  PicoSeach 


Instructional  design  theories  included  in  the  database:  (follow  links  for  more  information) 

Algo-Heuristic  Theory 

Cognitive  Load  Theory  and 

4C-ID  Model 

Minimalism 

SOI 

Figure  1 - A screen  shot  from  the  Instructional  Design  Database 
(http  ://web  .cortland.edu/ frieda/ID/IDdatabase.html). 


During  the  process  of  reviewing  the  design  theories  and  models,  the  students  were 
able  to  define  the  fields  for  the  database.  The  fields  that  were  developed  fit  into  three 
categories  based  on  the  working  definition  of  instructional  design  theories:  1 . theory 
basics  (theory  name,  author  name,  associated  learning  theory,  and  model  description),  2. 
theory  specifics  (goals  or  preconditions,  principles,  condition  of  learning,  required  media, 
role  of  facilitator,  instructional  strategies,  assessment  method),  and  3.  formative  research 
and  application  (tested  context,  research  method,  research  conditions,  resources).  This 
was  a difficult  process  because  the  class  had  to  define  the  outcomes  (the  fields)  of  their 
process  before  they  were  experts  on  the  information.  This  work  did,  however,  compel  the 
students  to  grapple  with  the  content  at  a much  deeper  level  than  if  the  fields  already 
existed  and  the  students  just  had  to  fill  them  in. 

Perhaps  the  hardest  part  of  the  entire  project  was  defining  the  potential  queries  of 
the  database.  This  was  the  last  step  undertaken  in  the  design  of  the  database,  which  is 
perhaps  evident  upon  trying  to  query  the  database.  Because  the  potential  queries  were 
saved  for  last,  there  was  a lack  of  uniformity  in  the  records  which  made  the  database  hard 
to  search.  Because  of  this,  the  nature  of  the  finished  product  is  less  like  a true  database 
and  more  like  an  indexed  web  search.  This,  however,  informed  the  students  of  how  to 
better  design  the  database  in  a next  attempt. 

Generative  Practice  Project:  Creating  a Generic  Development  Model 
In  the  beginning  of  the  seminar  course,  Yongjin  Lee  started  with  a fuzzy  question:  what 
is  constructivist  instructional  design  and  development?  From  literature  review  and 
discussion  with  colleagues,  she  acknowledged  the  influence  of  different  epistemological 
perspectives  (i.e,  objectivism  vs.  constructivism)  on  the  formation  of  instructional  design 
theories.  As  the  epistemological  assumptions,  constructivism  and  objectivism  should  be 
considered  as  two  alternatives  on  learning  and  understanding  (Cunningham,  1992; 
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Hwang,  1996).  The  two  theoretical  positions  can  be  complementarily  applied  in 
instructional  design  practices  (Hwang,  1996),  but  the  epistemological  perspectives  can 
not  be  mixed  in  an  instructional  program  (Bednar,  Cunningham,  Duffy,  & Perry,  1995). 
Instructional  designer  may  select  a theoretical  position  to  optimize  the  learning  and 
instruction  situation  within  each  given  setting  and  condition.  Although  the  instructional 
design  theories  could  provide  generic  and  flexible  prescriptions  to  any  learning  situation, 
making  of  instructional  decision  depends  upon  the  instructional  designer’s  analysis  on  the 
given  situation.  Instructional  design  practice  is  a form  of  situated  activity  depending  on 
the  specific,  concrete,  and  unique  circumstances  of  the  project  (Streibel,  1995).  Based  on 
the  knowledge  of  instructional  design  theories,  the  instructional  designer  should  be  able 
to  select  instructional  strategies  to  be  called  upon  for  the  specific  conditions  of  learning 
(Merrill,  1992). 

The  constructivist  theories  assume  that  our  knowledge  of  the  world  arises  through 
our  construction  and  generation.  Learners  construct  knowledge  and  subjectively  process 
it  in  ways  that  reflect  their  needs,  prior  experiences,  and  attitudes.  For  the  constructivist, 
learning  is  problem  solving  based  on  personal  discovery  and/or  socially  generated 
interpretation  of  experiences.  Thus,  constructivist  instructional  designers  focus  on  the 
instructional  methods  and  strategy  that  help  learners  actively  explore  topics,  advanced 
their  thinking,  and  develop  their  own  interpretation  of  knowledge  (Jonassesn,  1999). 

In  regard  with  generating  an  instructional  development  model,  Lee  has  developed 
a functional  ID  model  based  on  constructivist  approach.  She  conceptualized  that 
Instructional  Development  (ID)  model  indicates  systematic  processes  for  developing  and 
managing  instructional  methods,  strategies,  and  media,  in  order  to  attain  the  instructional 
goal  in  the  given  setting.  Her  functional  ID  model  is  designed  to  articulate  generic  actions 
and  strategies  to  develop  constructivist  learning  environments  in  a variety  of  educational 
settings.  The  model  consists  of  five  focal  points  including  Analysis,  Design,  Evaluation, 
Development,  and  Implementation.  Each  focal  point  is  not  regarded  as  a sequential  stage 
of  instructional  design  activities,  but  as  an  aspect  of  primary  action  and  consideration 
which  instructional  designer  must  focus  on  to  accomplish  whole  ID  process.  This  non- 
linear ID  approach  stems  from  constructivist  ID  models  (i.e.,  Crawford,  2004;  Willis 
1995),  which  emphasizes  real-world  focus  upon  the  evolving  process  of  analysis,  design, 
development,  and  implementation,  through  formative  evaluation  and  feedback  from  the 
end  users  or  students. 

For  the  last  two  decades,  linearity  of  the  Instructional  System  Design  (ISD) 
models  has  been  debated,  as  growing  interest  towards  constructivism  in  ID  field  (Willis, 
1995).  Since  the  ID  practice  is  a complex  and  dynamic  system,  the  linear  ISD  approaches 
have  limits  to  encompass  the  dynamic  processes  and  changes.  Lastly,  constructivist 
instructional  development  models  try  to  employ  non-linear,  flexible  and  authentic  view  of 
the  ID  process  (Crawford,  2004;  Willis,  1995).  The  recent  constructivist  ID  models  such 
as  Willis’  and  Crawford’s,  however,  mostly  focus  on  conceptual  modeling  which  intends 
to  describe  just  taxonomies  or  components  considered  in  the  ID  process.  There  seems  a 
lack  of  explanation  of  practical  and  generic  action  plan  and  strategies  of  the  ID  process 
from  the  constructivist  view.  Within  the  magnified  flexibility  of  the  new  constructivist  ID 


155 


models,  instructional  designer  should  take  a huge  burden  of  selecting  and  elaborating 
each  specific  task  and  specifying  action  plans.  The  functional  ID  model  focuses  on 
specifying  the  recursive  and  interrelated  procedures  across  five  focal  points  as  well  as 
offering  conceptual  model  of  the  ID  process.  The  model  intends  to  describe  what  and 
how  instructional  designer  is  doing  in  the  ID  process,  through  the  explanation  of  primary 
concerns  and  the  interrelationship  of  the  concerns. 

Four  primary  concerns  include:  problem,  task,  resource,  and  communication.  The 
Analysis  focus  starts  with  problem  in  the  given  setting.  The  problem  is  related  to  a gap 
between  the  present  status  and  the  ideal  future  scenario  of  the  instructional  system.  ID 
process  is  a sort  of  problem  solving  performance  to  resolve  the  deficiency  in  the  given 
system.  Instructional  designer  should  identify  problem  symptoms  in  the  system  through 
inductive  analysis  of  performances  and  environment  analysis  (Tessmer,  1990;  Wilson, 
Jonassen,  & Cole,  1993).  Also,  the  problem  is  considered  as  the  questions,  the  case,  or 
the  project  that  learners  attempt  to  solve  or  resolve  (Jonassen,  1999).  This  ID  model 
assumes  that  instructional  design  must  focus  on  active,  constructive,  and  authentic 
learning  with  problem-based  tasks  across  curriculum.  Instructional  designer  must  analyze 
the  variety  of  expert  users  and  their  performance  in  order  to  select  the  problems  driving 
learning.  Inductive  performance  analysis  such  as  critical  incident  analysis  (Jonassen, 
1987;  Jonassen,  Tessmer,  & Hannum,  1999)  can  be  considered  at  that  point. 

Within  the  Design  phase,  instructional  designer  should  consider  what  and  how 
tasks  are  accomplished.  The  instructional  designer  must  identify  the  tasks  that  the  variety 
of  expert  users  do,  rather  than  define  isolated  unit  of  content  and  type  of  learning  with 
predetermined  objectives  (Bednar,  Cunningham,  Duffy,  & Perry,  1992).  The  instructional 
designer  identifies  authentic  tasks  and  provides  instruction  in  the  context  of  those  tasks 
(i.e.,  Sticht  & Hickey,  1988).  Constructivist  learning  environment  is  designed  to  provide 
a rich  context  in  which  students  are  exposed  to  multiple  perspectives  and  can  learn  to 
construct  multiple  perspectives  on  an  issue  (Bednar,  Cunningham,  Duffy,  & Perry,  1992). 
To  build  such  a learning  environment,  it  is  essential  to  design  a variety  of  instructional 
strategies  such  as  collaborating,  coaching,  simulating,  evaluating,  and  scaffolding. 
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Figure  2 - A visual  representation  of  the  Constructivist  Instructional  Development 

Model  (created  by  Y.  Lee). 

The  development  focus  is  closely  related  to  how  to  manipulate  and  adopt  relevant 
resources  for  developing  a rich  learning  environment.  Resources  may  include 
information,  technology,  and  human,  which  are  required  to  design  and  develop 
instructional  materials  and  tasks  that  are  appropriate  for  facilitating  learners  to  construct 
their  knowledge.  Development  team  designs  and  develops  instructional  materials  and 
tasks  based  on  the  selected  resources,  and  makes  revisions  to  them  with  ongoing 
formative  evaluation.  Even  though  this  model  depicts  Design,  Development,  and 
Evaluation  as  separate  focal  points,  the  process  of  design,  development,  and  evaluation 
mostly  occurs  at  the  same  time.  This  is  because  formative  evaluation  and  revision  would 
occur  from  initial  design  and  development  phases,  and  reflection  and  adjustments  in 
design  and  development  would  happen  recursively. 

In  particular,  the  ID  model  places  the  Evaluation  focus  in  the  center  core  of  the  ID 
process  because  evaluation  can  occur  and  make  changes  at  any  stage  of  ID  process.  The 
ID  model  emphasizes  formative  evaluation  which  provides  feedback  the  ID  team  can  use 
to  improve  and  enhance  the  instructional  media  and  tasks.  Revision  is  directly  led  based 
on  formative  evaluation  feedbacks  across  the  five  focal  points.  Specifically,  students  and 
expert  users  frequently  evaluate  any  components  of  the  instructional  materials  at  various 
stages,  and  give  feedback  which  might  be  related  to  primary  concerns  such  as  problems, 
tasks,  resources,  and  communication. 

To  develop  and  implement  high  quality  of  instructional  products,  communication 
among  stakeholders  is  one  of  the  most  important  concerns.  ID  team  may  develop 
instructor  guides,  self-paced  tutorial,  interactive  computer-based  training,  participant 
workbooks,  operations  manuals,  or  job  aids  in  order  to  inform  the  instructional  materials 
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and  tasks  developed.  Specifically,  instructional  designer  should  utilize  efficient  and 
multiple  channels  to  inform  the  design  and  development  of  learning  environments  and  to 
receive  feedback  from  potential  users  and  stakeholders.  Besides,  communication  is  a 
critical  component  for  collaborative  development  process.  Since  the  CID  model 
encourages  the  collaborative  development  environment  in  which  a variety  of  specialists 
related  to  ID  and  the  end-users  participate,  selection  of  efficient  communication  strategies 
would  be  considered  at  every  focal  point  of  ID  process. 

In  sum,  the  functional  ID  model  consists  of  five  focal  points  involving  Analysis, 
Design,  Evaluation,  Development,  and  Implementation,  which  is  a non-linear  and 
recursive  process  with  an  emphasis  on  four  concerns:  problems,  tasks,  resources,  and 
communication.  Ongoing  formative  evaluation  would  review  the  primary  concerns  at  any 
stage  of  ID  process  and  drive  the  instructional  designer  to  the  next  step  and  revision 
across  five  focal  points.  Particularly,  the  ID  model  emphasizes  inductive  analysis  process 
of  problems  and  tasks,  and  participation  of  the  end-users  throughout  ID  process.  At  the 
end  of  the  course,  Lee  emphasized  that  this  generative  project  of  ID  model  allowed  her  to 
analyze  and  synthesize  moderate  principles  provided  by  constructivist  instructional 
design  theories  in  order  to  develop  practical  heuristics  and  work  plans  for  instructional 
development. 

Conclusion 

The  use  of  a hands-on  and  active  process  engaged  the  seminar  students  and  compelled 
them  to  a deeper  understanding  of  the  course  content.  The  differences  between 
instructional  design  and  development  are  not  easily  recognizable,  especially  with  the 
confusion  about  the  tenninology  used  seemingly  interchangeably  in  the  literature.  During 
this  intensive  and  deep  look  at  the  content,  by  examining  the  literature  of  prominent 
members  of  our  field,  by  working  in  depth  through  a database  project,  by  creating  and 
justifying  an  instructional  development  model,  the  students  in  the  seminar  gained  a much 
deeper  understanding  of  their  field. 
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Paperless  Accreditation: 

NCATE  is  Ready.  Are  We? 

Jerry  P.  Galloway 
Georgia  Southern  University 

This  report  outlines  a case  of  a Upper-Midwestern  School  of  Education’s  2-year  preparation  for  an 
accreditation  visit  and  their  efforts  to  develop  a completely  electronic  document  center.  The  School  of  Education 
(SOE)  is  an  NC ATE-accredited  (National  Council  for  Accreditation  of  Teacher  Education,  2005)  university  that 
prepares  K-12  and  graduate  educators  for  licensure  in  a Midwestern  State.  Many  concerns,  obstacles  and  basic 
principles  have  been  made  clearer  by  this  endeavor  and  this  report  outlines  those  issues  in  some  detail. 

Electronic  accreditation  review  is  still  a new  idea  and  not  yet  embraced  by  the  majority  of  schools.  Any 
management  of  electronic  media  beyond  one’s  own  files  on  an  individual  machine  can  require  a larger  infrastructure 
in  support.  It  is  certainly  not  an  easy  transition  and  what  is  required  or  how  to  proceed  is  not  self-evident.  This 
report  of  one  such  case  will  hopefully  offer  some  ideas  for  improving  the  process  and  probably  of  success  for 
schools  considering  this  option.  One  day  very  soon,  perhaps,  an  electronic  review  will  be  the  accepted  norm. 

For  an  accreditation  review,  of  course,  the  document  center  ideally  provides  all  of  the  appropriate 
documentation  to  support  excellence  throughout  the  program.  Considering  the  number  and  variety  of  documents 
suitable  to  this  challenge,  any  given  document  center  must  certainly  be  unique  to  the  institution  and  the  time.  This 
report  does  not  address  the  problems  and  difficulties  or  successes  that  may  have  arisen  in  identifying  or  gathering 
that  documentation.  This  report  will  focus  more  specifically  on  the  electronic  issues  and  nature  of  the  product. 


What  is  an  Electronic  Document  Center? 

Similar  to  an  electronic  portfolio  for  teachers  (Aschermann,  1999),  the  document  center  is  the  portfolio  for 
the  institution.  Artifacts  are  collected  and  organized  to  demonstrate  and  document  the  performance  and 
achievements  of  the  institution.  Standards  that  govern  the  program  can  provide  a framework  for  arranging  and 
presenting  the  contents  of  the  document  center  (NCATE,  2002;  NCATE,  2003).  While  ten  years  ago  an  electronic 
accreditation  review  may  have  been  an  experimental  novelty  (Polis,  1994),  today’s  trend  is  toward  an  electronic 
document  center  (EDC)  (Salzman  & Zimmerly,  2002;  NCATE  2005)  although  still  not  required  by  NCATE. 
Essentially,  an  EDC  means  that  both  the  artifacts  themselves  are  electronic  (word  processed  documents,  PDF 
documents,  graphic  files,  web  site,  digitized  video  clips,  etc.)  and  the  delivery  mechanism  or  access  system  is 
electronic.  Review  and  examination  of  materials  throughout  the  document  center  (DC)  is  then  done  via  computer. 

It  can  be  considered  somewhat  incidental  whether  the  EDC  is  in  fact  on-line  (accessible  through  the 
Internet)  or  instead  viewable  on  a local  hard  disk,  CD-ROM  or  other  media,  or  both.  As  you’ll  read  below,  there  are 
issues  affecting  the  delivery  mechanism  but  these  are  still  peripheral  to  whether  a DC  is  electronic  or  not.  In  an 
ideal  EDC,  no  paper  is  used.  Many  faculty  may  still  see  a VHS  tape  as  technology  but  it  is  at  least  an  outdated 
medium  and  video  material  should  ideally  be  made  available  through  the  same  computer  interface  as  other  media. 
Materials  are  therefore  stored  in  an  electronic  format  either  from  their  original  creation  or  converted  by  scanning  or 
digitizing  the  tactile  or  paper  media  into  an  electronic  format.  These  fdes  are  then  arranged  and  stored  electronically 
on  a computer. 

There  are,  then,  numerous  obstacles  and  issues  involved  in  creating  a complete  EDC.  The  production  of 
materials  in  appropriate  media  formats  is  only  one  hurtle.  The  development  and  maintenance  of  the  delivery 
mechanism,  a modern-style  hyper-media  delivery  system,  as  well  as  the  management  of  both  the  site  and  the 
accreditation  project  are  important  concerns.  A delivery  or  access  system  is  necessary  for  viewing  the  materials. 
With  hundreds,  if  not  thousands,  of  files  they  must  be  organized  in  a manner  that  makes  sense  to  the  accreditation 
review  team  and  are  browsed  easily  and  efficiently  by  most  anyone.  This  might  very  likely  be  the  Internet  and  an 
Internet  browser  tool  (like  Netscape  or  Microsoft’s  Internet  Explorer)  but  not  necessarily.  Other  tools  can  of  course 
be  developed  if  a programming  staff  is  on  hand  to  write  an  appropriate  software  interface  but  this  is  not  usually 
available.  So,  the  most  readily  available  tool  for  hyper-media  is  often  an  Internet  browser.  Whether  the  EDC  is 
actually  placed  on-line  can  be  considered  incidental. 
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EDC  Director 

In  this  case,  the  EDC  Director  was  the  technology  professor  - a one -person  department  - because  no  one 
else  was  technically  literate.  With  consultation  with  SOE,  the  EDC  Director  developed  an  Internet-based  system  to 
be  accessed  by  the  standard  browser  tools  mentioned  above.  The  site  was  also  made  available  on  both  Macintosh 
and  Windows  platforms.  To  design  and  develop  an  accreditation  document  center,  it  has  been  thought  that  a 
specialist  may  be  unnecessary  and  that  there  may  be  many  individuals  at  an  institution  suitable  for  the  task  (Zelazek 
& Garten,  1999).  But,  an  electronic  document  center  (EDC)  involves  other  issues  and  a technical  specialist  may 
indeed  be  necessary.  More  on  the  role  and  responsibilities  of  the  EDC  Director  will  be  discussed  below. 


Technical  - The  Overall  Format 

The  dual-platform  feature  of  the  EDC  was  not  a difficulty  as  the  Internet  is  already  dual-platform.  Making 
the  EDC  like  a giant  website,  whether  on-line  or  not,  and  accessible  with  a browser  tool  already  allows  for  both  Mac 
and  PC  access.  Basically,  the  SOE  gathered  materials  in  electronic  format  and  converted  materials  not  already  in 
electronic  format  by  scanning  them  into  a computer,  and  created  a series  of  web  pages  with  hyperlinks  for  accessing 
those  artifacts.  Use  of  the  EDC  can  then  be  just  like  using  virtually  any  web  site. 

The  cross-platform  nature  of  HTML  web  pages  can  solve  some  problems.  But,  there  are  other  issues  about 
conflicts  between  the  platforms  that  will  be  addressed  below. 

In  this  reported  case,  the  SOE  used  exclusively  a PC  with  Windows  XP  for  developing  and  managing  the 
EDC.  About  50  percent  of  the  faculty  personally  used  Macintosh  machines  for  their  own  work  in  developing  and 
submitting  documents  to  the  DC. 

The  old  programmer’s  adage  that  the  program  must  be  “user-proof’  is  very  applicable  here  as  the  EDC 
must  allow  for  reviewers  who  may  not  be  particularly  computer-literate  or  even  competent  with  technology.  It 
would  be  beneficial  for  the  user-interface  to  be  designed  in  a way  that  allows  the  simplest  operation  with  maximum 
visual  clarity.  The  design  hopefully  promotes  meaningfulness  for  the  standards  or  principles  that  the  DC  represents; 
this  must  be  clear  to  the  accreditation  reviewers.  This  means,  among  other  things,  that  a simple  point-&-click 
interface  must  be  readily  available  throughout  the  EDC  pages. 

Figure  1 illustrates  the  style  of  how  documents  are  presented  on  the  screen  in  the  document  center  (EDC). 
Figure  1 shows  only  a segment,  of  course,  of  an  elaborate  series  of  Internet  pages  for  the  various  programs  and 
standards  and  elements  (sub-standard  #1)  that  must  be  addressed  in  the  EDC.  Note  the  standards  that  are  described 
(in  bold)  as  distinct  from  the  actual  artifact  descriptions  (a,  b,  etc.)  which  inform  the  reviewers  about  what  the 
document  is  and  why  it  is  presented  in  evidence.  The  document  is  accessed  simply  by  clicking  the  “CLICK  HERE” 
hyperlink. 


I A 

PROGRAM  AREA 

STANDARD  - IA:  Statement  of  Standard 

A - #1 

Description  of  the  specific  standard  being  address  goes  here.  Sometimes  this  is  one 
sentence  but  may  involved  two  or  three. 

Indicator 

Description 

LINK 

a 

Artifact  description  stating  what  the  document  is. 

CLICK  HERE 

b 

Description  indicating  the  document's  purpose  here. 

CLICK  HERE 

Figure  1.  Sample  segment  of  EDC  artifacts  accessible  via  “CLICK  HERE”  links. 


Figure  2 is  intended  as  a slight  modification  on  the  EDC  layout  and  design  shown  in  figure  1 above.  The 
primary  difference  is  simply  that  the  hyperlinks  are  informative  and  indicate  the  kind  of  document  being  accessed. 
This  can  be  useful  for  the  accreditation  reviewers  in  addition  to  the  EDC  project  coordinator  so  they  will  know  what 
application  will  be  invoked  by  clicking.  This  can  not  only  contribute  to  some  peace  of  mind  (knowing  what  the 
computer  is  trying  to  do)  but  can  also  be  helpful  in  trouble-shooting  any  failed  links. 
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Figure  2.  Alternative  sample  showing  informative  artifact  hyperlinks  (red  arrow). 
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Regardless  of  the  nature  of  the  hyperlink,  the  program,  standards  and  elements  should  be  distinguished  by  color 
shading  and  the  use  of  colored  text.  Color,  as  a navigational  clue,  can  be  as  important  and  useful  as  text  labels,  if 
not  more  so.  Color  can  establish  a theme  that,  after  brief  exposure,  will  help  to  familiarize  the  reviewer  with  the 
particular  area  being  viewed  or  assist  in  finding  another. 

Figure  3a  illustrates  this  use  of  color  in  a black  and  white  medium  by  shading  the  rows  with  and  setting  the 
text  as  black  or  white  in  contrast  to  the  shading.  The  colors  throughout  a given  standard  (I,  II,  III,  etc.)  should  be 
consistent  and  differ  from  colors  used  in  another  standard. 

Figure  3b  illustrates  the  use  of  color  to  distinguish  between  the  various  standards  and  program  areas.  The 
colors  throughout  a given  standard  (I,  II,  III,  etc.)  should  be  consistent  and  differ  from  colors  used  in  another 
standard. 
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STANDARD  - LA:  Statement  of  Standard 

A - #1 

Description  of  the  specific  standard  being  address  goes  here.  Sometimes  this  is  one 
sentence  but  may  involved  two  or  thr  ee. 
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LINK 
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Artifact  description  stating  what  the  document  is. 

Word  File 
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Description  indicating  the  document’s  purpose  here 

PDF  File 

Figure  3 a.  Illustration  of  color  by  shading  in  black  and  white. 
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Figure  3b.  Illustration  of  the  use  of  different  colors  by  standard  and  program  area. 
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High  contrast  between  text  and  background  shading  will  improve  clarity  and  make  it  easier  to  read.  Of 
course,  which  colors  are  used  is  entirely  an  arbitrary  choice  as  there  are  no  accepted  color  themes  for  educational 
standards,  NCATE  or  otherwise. 

In  general,  overall  display,  which  includes  the  text,  the  colors,  the  table-layout,  etc.,  can  be  created  using  a 
simple  web  page  composer  tool.  This  can  include  something  as  simple  and  straight  forward  as  the  Netscape 
Composer  tool  or  the  more  involved  Microsoft  FrontPage  composer  or  DreamWeaver.  The  specific  composer  tool 
doesn’t  really  matter  and  will  inevitably  require  some  basic  knowledge  of  HTML  so  that  the  underlying  code  can  be 
occasionally  manipulated  manually  and  directly  using  a simple  editor. 

The  electronic  documents  should  be  segregated  into  separate  folders,  one  for  each  standard.  The  hyperlinks 
will  also  be  individually  created  once  the  artifact’s  Indicator  and  Description  are  completed  in  the  EDC  and  the 
electronic  document  is  available  in  the  corresponding  folder.  Figure  4 illustrates,  with  a very  limited  scope,  the 
hierarchical  relationship  among  EDC  web  pages  and  corresponding  folders  functioning  as  containers  for  the  EDC 
artifacts. 


Figure  4.  Abbreviated  illustration  of  the  hierarchical  relationship  within  the  EDC.  Artifacts  are  stored  in 
folders  by  standard  and  element. 


The  artifacts  are  stored  in  the  various  folders  and  are  accessed  from  the  web  page  for  that  standard.  Also,  in  figure 
4,  the  entire  assembly  represents  part  of  only  one  program  area  such  as  a typical  undergraduate  program  or  perhaps  a 
Masters  Degree  program,  etc.  The  same  table  of  contents  page  shown  at  the  top  might  also  lead  to  each  of  the 
various  program  areas  or,  instead,  intervening  menus  might  be  provided  depending  on  how  it  is  setup. 

Once  the  artifacts  are  arranged  into  the  folders  and  the  corresponding  EDC  web  page  is  linked  to  it,  then 
that  HTML  file  and  the  artifact  file(s)  will  need  to  be  uploaded  (“posted”)  to  the  EDC’s  Internet  site.  Any  freeware 
FTP  program  (file-transfer  protocol)  is  sufficient  and  recommended  for  this  step  in  the  procedure. 

It  is  not  recommended  that  one  use  an  automatic  “publishing”  from  within  a composer  tool  as  the  means  for 
posting  documents  to  the  web.  This  process  tends  to  hide  the  details  of  the  transaction  from  view.  Automated 
publishing,  is  viewed  by  many  as  a convenience  but  seems  to  purposefully  automate  the  task  for  those  who  do  not 
wish  to  or  are  not  able  to  understand  well  enough  to  manage  it  directly.  A simple  FTP  program  will  expose  the 
process  and  allow  one  to  view  and  manage  all  files  and  folders  clearly  and  directly  at  the  web  site. 

Technical  - Generating  Electronic  Documents 

While  building  the  EDC  web  pages  and  “posting”  the  artifacts  to  the  Internet  is  a long  and  tedious  part  of 
the  process,  the  production  of  electronic  artifacts  is  no  less  involved.  Generally,  faculty  with  the  assistance  of  staff 
will  gather,  collect,  generate  and  produce  the  various  artifacts  for  the  DC.  They  can  then  be  submitted  to  the  persons 
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involved  in  developing  the  EDC.  The  separation  or  arrangement  of  duties  and  roles  for  those  involved  in  the 
development  of  the  EDC  will  be  discussed  further  below.  In  any  event,  support  staff  will  be  necessary  both  to  aid 
faculty  in  document  collection  and  preparation  and  also  to  serve  directly  in  building  the  EDC. 

A variety  of  file  formats  will  be  likely  as  it  was  in  this  case  for  the  SOE.  Obviously,  it  would  be  best  to 
select  a single  word  processing  format  to  use  in  the  EDC,  consistent  with  the  utility  that  will  be  available  on  the 
machines  used  by  the  visiting  accreditation  team.  The  SOE  utilized  three  primary  formats:  word  processed 
documents.  Acrobat  Reader  documents  commonly  called  PDF  documents,  and  the  URL’s  (addresses)  of  existing 
web  sites.  To  a lesser  extent,  raw  graphic  files  were  also  included.  It  is  also  reasonable  to  anticipate  the  use  of  both 
plain  text  and  graphic  files. 

Often,  documents,  having  been  created  in  an  electronic  medium  such  as  word  processing,  are  still  available 
in  that  format  and  can  be  submitted  directly  in  their  original  form.  Some  faculty  may  use  different  applications  and 
so  documents,  even  though  word  processed,  may  exist  in  various  formats  such  as  Microsoft  Word  (doc),  Word 
Perfect  (wpd),  Microsoft  Works  (wps),  or  other.  It  is  not  really  practical  to  attempt  to  convert  all  artifacts  into  the 
same  format. 

Focus  and  Highlight 

Half  way  through  document  collection  it  was  determined  that  more  should  be  done  to  target  or  highlight 
certain  material  in  order  to  focus  the  accreditation  team  to  specific  areas  of  artifacts.  This  is  not  an  easy  thing  to  do 
in  an  electronic  document  system.  If  the  target  is  a web  site  then  a specific  page  URL  within  the  site  could  be  used. 
If  that  page  has  an  anchor  or  named  location  within  it  then  it  may  be  possible  to  link  directly  to  an  internal  part  of 
that  web  page  and  thus  focus  on  a highly  specific  area.  Otherwise,  one  must  likely  accept  and  use  a web  site  the 
way  one  finds  it. 

For  documents  generated  within  the  school,  there  is  some  control.  The  SOE  decided  to  use  a yellow 
highlighter  to  literally  highlight  sections  of  paper  documents  in  order  to  focus  attention  on  that  material.  Scanning 
in  color  then  maintained  the  highlighted  color.  For  documents  still  in  electronic  format,  the  highlighting  feature  of 
Microsoft  Word  allowed  some  similar  highlighting  of  specific  material  but  not  everyone  used  this  feature. 

In  any  event,  linking  within  the  EDC  to  a specific  document  still  does  not  necessarily  place  the  highlighted 
area  within  the  initial  screen  view.  Bookmarks  within  Word  documents  can  function  much  like  anchors  or  named 
locations  in  web  pages  and  thereby  allow  direct  linking  to  specific  sections  of  material.  But,  this  would  be  even 
more  time  consuming  and  require  specific  skills  not  available  for  most  faculty.  So,  this  option  was  not  used  at  all 
and  the  problem  of  focusing  attention  on  specific  areas  of  artifacts  remained.  The  easiest  solution  was  to  include 
some  direction  in  the  artifact  description  to  better  direct  the  accreditation  team. 

Technical  - Production  and  Process 

A dedicated  computer  (PC)  was  used  as  the  primary  work  station.  It  is  imperative  to  maintain  one  primary 
work  site  to  avoid  duplications  and  gaps  between  conflicting  work  stations.  Netscape  Composer  was  used  80%  of 
the  time.  Less  than  5%  of  updates  or  changes  were  done  with  Microsoft’s  FrontPage  but  specialty  features  of 
FrontPage  were  avoided  to  keep  the  HTML  code  clean  and  constructed  simply.  The  other  15%  of  the  time,  the 
HTML  code  was  manipulated  directly  using  Microsoft  WordPad. 

Submissions 

The  EDC  Director  prepared  specific  systems  to  help  the  SOE  submit  materials.  One  method  was  a 
dedicated  email  address  so  documents  could  be  submitted  as  attachments  to  emails.  The  faculty  was  basically 
familiar  with  how  to  do  this  so  this  proved  to  be  a convenience.  The  artifact  attachments  went  directly  to  the  project 
email  address  and  could  be  accessed  at  the  work  station. 

A web  site  was  developed  that  outlined  submission  procedures  in  detail.  The  SOE  faculty  were  acquainted 
with  the  page  through  a number  of  workshop  sessions.  In  spite  of  this,  throughout  the  first  year  and  half  most  did 
not  use  and  were  generally  unfamiliar  with  the  prescribed  submission  process.  Overall,  the  faculty  was 
inexperienced  with  or  not  well-versed  in  the  use  of  technology,  how  Internet  browsers  work,  and  the  subtleties  of 
URL’s,  passwords,  bookmarks  and  more. 

All  submissions  were  to  include  specific  information  required  to  complete  the  artifact  posting  and  for 
monitoring  submission  traffic.  As  mentioned,  workshops  provided  instructions  and  hardcopy  fliers  provided 
procedures.  For  tracking  artifacts  through  the  submission  and  posting  process,  the  web  site  provided  an  on-line 
document  submission  form.  Another  option  included  cut-n-paste  text  that  could  be  added  to  the  body  of  an  email 
submission.  Thirdly,  one  could  print  a PDF  form  for  submitting  the  information  with  any  paper  submissions. 
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In  spite  of  this  support  system,  the  first  year  included  countless  errors  and  critical  omissions  that  caused 
errors  and  generally  slowed  the  process.  Submissions  in  error  were  returned  for  correction  and  posting  was  slow. 

As  deadlines  neared,  faculty  began  to  learn  the  process  and  postings  progressed. 

Files 

One  problem  area  among  many  was  the  submission  of  artifacts  with  poor  or  useless  filenames.  Filename 
problems  included  at  least  the  following: 

a.  missing  filename  extensions  (using  filename  stem  only) 

b.  the  use  of  more  than  one  period  in  the  filename 

c.  the  inclusion  of  spaces  within  the  filename 

d.  a mix  of  capital  and  lowercase  letters 

e.  extra  long  filenames 

The  last  problem  was  exemplified  by  one  file  submission  with  more  than  70  characters  in  the  filename  stem.  This 
extreme  was  not  an  oversight  and  was  not  viewed  as  unusual  by  the  person  submitting  the  file. 

Another  problem  was  the  general  over  sizing  of  many  files  as  a result  of  scanning.  A paper  document 
might  be  placed  on  a scanner  and  converted  to  a graphic  image.  Scaled  to  view  on  the  screen  the  faculty  person  or 
their  assistant  doing  the  work  may  not  recognize  that  the  file  is  actually  much  larger.  That  is,  a “page”  may  be  20  by 
25  inches  or  so  and  yet  still  appear  to  be  only  4 by  5 inches  because  of  the  smaller  viewing  area.  The  image  should 
be  resized  so  that  4 by  5 inches,  for  example,  is  the  full  100%  size.  To  be  more  technical,  the  file  may  be  15mb 
when  it  only  needs  to  be  800K.  Faculty  and  staff  had  no  notion  of  file  sizes  and  no  knowledge  of  how  to  modify 
such  attributes.  Continuing  this  problem  across  several  thousand  documents  can  create  a very  big  problem.  This 
issue  will  be  discussed  further  in  the  management  section  below  but  in  this  case  the  SOE  retained  the  many 
oversized  files.  Once  the  problem  had  spread  throughout  the  many  electronic  materials,  fixing  the  problem  was  no 
longer  a real  option. 

Overall,  the  extreme  lack  of  technical  literacy  on  the  part  of  the  SOE  faculty  contributed  greatly  to  the 
many  problems  of  getting  materials  submitted  and  posted  efficiently.  Flaving  only  one  educational  technology 
professor  (EDC  Director),  was  not  a sufficient  compensation  for  a considerable  lack  of  techno-literacy  among  the 
other  faculty,  staff  and  administration. 

Backups 

Of  course,  any  electronic  file  system  must  include  a backup  system.  As  files  are  uploaded  daily  to  the  site, 
the  online  site  itself  functioned  as  the  first  level  of  backup.  That  is,  the  so-called  original  set  was  maintained  on  the 
work  station  computer,  so  the  online  version  was  essentially  a second  copy.  A third  backup  included  occasionally 
downloading  from  the  internet  version  to  an  office  computer  for  storage  there.  Lastly,  CD’s  were  occasionally 
burned  containing  the  complete  system  to  date. 

One  can  never  have  enough  backup  copies  so  we  recognize  that  this  system  included  some  risk.  For 
example,  the  only  up-to-date  file  sets  were  on  the  work  station  and  online.  The  second  machine  and  the  CD’s  were 
not  regularly  or  frequently  maintained.  If  the  work  station  had  crashed  then  the  online  version  would  have  been 
immediately  downloaded  to  update  the  second  computer.  Nevertheless,  this  constitutes,  from  a traditionally 
conservative  point  of  view,  a deficiency  during  the  EDC  development.  Numerous  backups  of  the  final  version  were 
prepared  on  CD-ROM. 

No  real  problems  were  encountered  in  this  case  due  to  insufficient  backups.  This  was  perhaps  due  as  much 
to  the  EDC  Director’s  availability  and  supervision  than  any  technical  support  mechanism. 

Posting  Files  Online 

Of  course,  files  have  to  be  uploaded  to  the  Internet  web  site.  This  is  typically  called  publishing  or  posting. 
Some  composer  tools  provide  an  automated  publishing  feature  and  allow  direct  editing  of  files  on  the  web  site.  That 
is,  one  can  grab  a file  directly  off  the  Internet,  modify  it  and  publish  directly  back  to  the  site  - all  in  one  process. 
Unfortunately,  this  can  confuse  or  confound  the  process  of  maintaining  the  original  files  in  the  departmental 
worksite.  Instead,  the  SOE’s  approach  was  to  work  directly  on  the  worksite  files,  simply  save  the  changes  to  hard 
disk,  and  then  upload  the  new  files  and  modifications  to  the  web  site.  Then,  with  the  use  of  a dedicated  FTP 
program  (file  transfer  protocol)  to  upload  the  files,  a greater  degree  of  clarity  was  maintained.  That  is,  an  FTP 
program  allows  one  to  examine  each  and  every  folder  and  file  of  the  web  site  directly. 
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Final  Format 

The  primary  use  of  the  EDC  was  directly  online.  The  home  page  URL  was  provided  to  the  reviewer  and 
they  were  able  to  browse  any  fdes  or  areas  as  desired.  The  SOE  provided  a conference  room  to  the  reviewers  and 
therein  a range  of  PC’s  and  Macintosh  computers  with  online  access  so  the  group  could  examine  the  EDC. 

In  past  accreditation  reviews,  computer  equipment  and  some  materials  were  provided  by  the  SOE  to  the 
review  team  at  their  hotel  for  after-hours  work.  In  this  case,  the  full  EDC  was  made  available  on  CD-ROM  disks  so 
the  entire  EDC  could  be  examined  off-line  directly  on  CD. 

Due  mostly  to  the  oversized  nature  of  many  files  resulting  from  oversized  scanning  (discussed  earlier)  the 
complete  EDC  was  larger  than  would  fit  on  one  CD-ROM  and  DVD-ROM  was  not  available.  The  EDC  was  split 
up  and  placed  on  different  disks  based  on  NCATE  standards. 

Platform  Conflicts 

Either  platform,  Macintosh  or  Windows  (PC),  was  able  to  access  the  EDC  online  as  the  primary 
presentation  medium,  HTML  Web  Pages,  is  already  cross-platform  compatible.  The  individual  files  consisting 
mostly  of  Microsoft  Office  formats  were  also  cross-platform. 

The  production  of  CD’s  was  done  on  PC’s  and  were  therefore  consistent  with  that  format.  However, 
burning  a CD  to  be  readable  by  either  format  was  deemed  unreliable.  That  is,  a CD  burned  on  a PC  might  not  be 
completely  functional  on  a Macintosh  because  of  a number  of  technical  reasons  not  discussed  here.  So,  the  entire 
EDC  was  transferred  to  a Macintosh  Computer  hard  disk  where  CD’s  could  be  created  specifically  for  the 
Macintosh  platform.  The  review  team  could  then  select  the  particular  CD-ROM  system  of  their  choice. 

Management 

The  planning  for  the  selection  of  artifacts  and  who  will  judge  their  evidentiary  value  for  accreditation  are 
outside  of  the  scope  of  this  paper.  Such  programmatic  considerations  are  typically  under  the  direction  of  the 
Department  Chair  or,  in  this  case,  the  Dean  of  the  SOE.  In  any  event,  it  is  sufficient  to  note  that  a school  will 
commit  to  a process  of  data  selection  and  collection. 

The  issue  is  what  to  do  with  the  documents  as  they  are  collected  or  designated  for  the  EDC.  The  document 
center  cannot  be  a kind  of  large,  public  “dipsy-dumpster”  that  permits  anyone  to  dump  their  materials  into  it.  Aside 
from  the  issues  of  controlling  and  directing  document  selection,  which  is  presumably  under  the  guidance  of  the 
program,  artifacts  must  be  technically  organized  and  stored  in  the  electronic  realm.  Getting  the  materials  from  the 
source  to  final  posting  is  a critical  path  and  managerial  issues  lie  along  the  entire  journey. 

A Collection  Center 

Artifacts  were  submitted  to  a central  repository  for  processing.  A singular  location  served  as  the  site  for 
gathering  documents  and  for  creating  the  electronic  media.  In  this  case  of  a web-based  EDC,  a dedicated  email 
address  served  as  the  conduit  to  a single  computer  used  to  store  submitted  materials.  The  machine  functioned  as  the 
primary  work  center  for  the  development  of  the  HTML  web  pages. 

A guiding  principle  in  the  development  of  the  website  was  to  create  a mirror  image  of  the  site  on  the 
workstation  computer.  Files  and  folders  were  posted  to  the  Internet  server  exactly  as  arranged  on  the  workstation. 
The  set  of  files  on  the  workstation  was  deemed  the  originals  and  all  backups,  copies,  and  postings  stem  from  there. 

It  can  be  important  to  have  a controlled  system,  a reliable  mechanism  for  tracking  submissions.  On 
numerous  instances,  discrepancies  arose  between  what  was  reportedly  submitted  and  that  which  was  posted. 
Ultimately,  materials  or  correction  instructions  had  to  be  resubmitted. 

Director/Coordinator 

The  one  person  designated  to  coordinate  and  technically  build  the  EDC  was  the  one  technology  professor 
in  the  SOE.  The  technical  responsibilities  of  such  a task  demanded  the  scrutiny  and  skills  of  the  technology 
specialist.  Nevertheless,  there  were  numerous  problems  with  this  role  as  discussed  below. 

Assistants 

Two  students  were  hired,  one  year  each,  to  assist  the  Director  throughout  the  two  year  development  of  the 
EDC.  Responsibilities  included  scanning  paper  materials,  retrieving  artifact  contributions  and  posting  documents  to 
the  EDC.  While  they  were  assigned  as  assistants  to  the  EDC  Director,  considerable  problems  arose  in  this  area  of 
management. 
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The  assistants  were  lightly  trained  with  no  prior  experience.  They  were  selected  more  because  of  their 
availability  than  their  aptitude  by  the  office  manager  (an  accreditation  officer)  and  the  dean.  In  addition  to  general 
EDC  development  and  management,  this  factor  made  close  contact  with  the  technology  director  all  the  more 
important.  However,  the  assistants  instead  reported  to  the  office  manager  rather  than  the  technology  director. 

Authority 

It  is  critical  to  have  a sound  and  respected  management  team  with  clear  authority  and  a clear  management 
structure  accepted  by  virtually  everyone.  The  office  manager  was  not  technically  knowledgeable  and  regularly 
misunderstood  the  specific  work  of  the  assistants.  The  assistants  performed  discrete  tasks  through  memorized 
procedural  rituals.  Nevertheless,  the  office  manager  seemed  to  value  this  role  and  acted  to  protect  it.  There  were 
many  problems  because  of  this  confusion  of  authority.  For  example,  on  more  than  one  occasion  the  office  manager 
distributed  unofficial,  unapproved  and  erroneous  announcements  to  the  department  declaring  the  EDC  status  and 
inviting  faculty  to  take  various  actions  with  submissions.  On  other  occasions,  the  office  manager  redirected  the 
work  load  of  the  assistants  with  serious  consequences  for  EDC  development. 

Even  when  this  problem  of  authority  and  control  was  questioned  and  later  challenged,  the  dean  failed  to 
enact  changes.  As  the  sole  university  administrator  with  any  enforceable  power  and  authority,  the  dean  really  sets 
the  stage  for  structure  or  chaos,  responsibility  versus  interference,  and  of  course  success  or  failure.  A great  deal  of 
good  will  seems  to  have  been  lost  in  this  and  other  conflicts  in  management.  The  technology  director  and  professor, 
originally  assigned  as  the  lead  responsibility  for  EDC  design  and  development,  was  not  empowered  to  carry  out  that 
role  without  considerable  conflict. 

With  extensive  experience  dealing  with  NCATE  reviews,  the  dean  naturally  served  as  the  leader  of 
document  center  content.  That  is,  prescribing  evidentiary  materials  that  would  be  necessary  and  appropriate  to 
support  all  aspects  of  the  program.  There  seemed  to  be  no  conflict  or  confusion  about  that  role  and  that  authority. 

Foresight  and  Planning 

Lack  of  training  and  knowledge  in  the  use  of  computer  technology  was  a considerable  problem  among  the 
faculty  and  staff.  Their  lack  of  sound  habits,  protocols  and  computing  discipline  carried  over  to  all  electronic 
material  management  and  production.  With  only  one  knowledgeable  technology  professor,  the  need  for  foresight 
and  planning  throughout  the  department  is  greater. 

Protocols,  procedures  and  various  tools  to  facilitate  the  development  and  submission  of  electronic  materials 
were  provided  and  outlined  by  the  EDC  director  at  the  beginning.  These  were  reviewed  in  subsequent  workshops 
and  maintained  throughout  the  2 year  process.  However,  factors  combined  to  undermine  the  effectiveness  of  these 
tools.  Faculty  ignored  the  protocols,  failed  to  understand  and  learn  procedures  and  generally  procrastinated  creating 
additional  work  for  the  director  and  causing  considerable  delay.  Additionally,  the  lack  of  a supported  authority 
structure  seemed  to  undermine  any  sense  of  urgency  or  legitimacy  in  the  technical  director’s  guidance  and 
leadership. 

Foresight  and  planning  can  be  the  critical  first  step  in  a successful  EDC  program.  "The  first  step  of 
providing  evidence  for  the  Continuing  Accreditation  on-line  sets  the  tone  for  preparing  documents  and  exhibits 
electronically.  " (Harry,  Brown,  & McCullogh,  2001,  p.  9).  For  the  SOE,  this  seemed  to  be  lacking  in  spite  of  an 
experienced  dean  and  a considerable  investment  in  an  independent  accreditation  consultant.  Certainly  a greater 
sense  of  team  identity  with  clearer  roles  would  have  greatly  improved  the  efficiency  and  quality  of  the  final  product. 
Perhaps,  better  trust  in  the  technical  advice  of  the  technology  expert  would  also  have  improved  both  the  process  and 
the  product. 


Final  Recommendations 

One  of  the  factors  missing  from  the  accreditation  visit  is  some  concern  for  the  amount  of  time  a given 
school  devotes  to  preparation  for  their  accreditation  visit.  If  school  A passes  with  near  perfect  results,  they  are  to  be 
commended.  If  school  B passes  with  a recognized  limitation  or  two,  they  may  be  pleased  but  plan  to  spend  more 
time  in  preparation  next  time. 

But,  what  if  school  A spent  literally  twice  as  many  faculty-hours  in  preparation  as  school  B?  More  to  the 
point,  school  A might  spend  many  thousands  of  faculty  hours  (as  was  done  in  this  case  study)  gathering  documents 
if  not  creating  documents  and,  in  a sense,  living  exclusively  for  the  sake  of  the  document  center.  This  is  contrary  to 
the  real  purpose  of  the  institution.  That  is,  the  purpose  is  to  deliver  a quality  educational  experience  and 
documenting  that  is  a secondary  mission. 
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The  analogy  that  seems  appropriate  is  how  a student  might  get  a very  high  score  on  a final  exam  by 
memorizing  all  of  the  answers  but  get  far  less  from  the  overall  course  experience  than  another  student  who  more 
literally  embodied  what  the  course  intended  yet  missed  a few  more  answers  on  the  test.  The  real  point  in  the  course 
is  better  represented  in  the  latter  student  who  will  continue  to  benefit  from  the  experience  in  the  future. 

There  are  problems  in  schools  focusing  on  the  accreditation  process  (Adams,  2004).  Schools  can 
erroneously  place  the  real  delivery  of  the  quality  educational  experience  behind  their  efforts  to  document  it.  The 
accreditation  process  should  consider  the  amount  of  faculty-hours  devoted  to  the  preparation  for  the  accreditation 
visit.  This  argument  is  based  on  the  premise  that  (a)  it  is  possible  to  compromise  the  quality  of  an  institution  and  it’s 
program  by  an  extreme  preoccupation  with  documentation  and  illustration  of  how  standards  are  being  met;  and  (b) 
still  conceal  or  fail  to  reveal  in  that  demonstration  the  preoccupation  and  exclusivity  in  creating  it. 

Finally,  a competent  technology  leader  should  be  available  to  direct  the  design  and  production  of  the  EDC 
and  should  be  given  the  authority  to  direct  the  use  of  that  technology.  The  entire  team  structure  should  be 
recognized  and  accepted  by  everyone  and  enforced  by  the  dean. 
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A Blended  Learning  Environment  as  Perceived  and  Experienced  by 

Course  Instructor 
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M.Ya§ar  Ozden 

Abstract:  Face-to-face  (FTF)  and  online  environments  require  different  skills  and 
techniques  for  an  instructor  to  employ  for  a lesson,  while  blended  learning  requires 
a harmony.  The  increased  research  on  the  conceptual  framework  and  design 
considerations  of  blended  learning  leads  to  a need  to  unfold  the  experiences  of  an 
instructor  in  the  design,  development,  and  implementation  processes  in  a blended 
learning  environment.  This  study  is  a preliminary  case  to  understand  the  context  of 
a blended  environment  based  on  instructor  experiences.  The  results  indicated  that 
blending  FTF  and  online  environments  require  thoughtful  design  and 
implementation  since  they  pursue  different  features  such  as  student  and  time 
management,  communication,  and  guidance. 

Key  Words:  Blended  Learning  Environments,  Instructor  Roles,  Instructor  Experiences. 

Introduction 

Blended  learning  has  the  potential  of  combining  best  of  two  worlds:  FTF  and  online  environments. 
With  this  feature,  it  is  gaining  more  attention  in  higher  education.  There  are  lots  of  issues  for  the  instructor  to 
take  into  consideration  while,  developing  the  course  for  both  in  FTF  and  online  environments.  Teaching 
methods,  instructional  approaches,  classroom  and  time  management  techniques,  communication,  motivation, 
resource  allocation,  required  technical  skills,  and  interpersonal  skills  are  a few  to  name. 

According  to  Driscoll  (2002),  blended  learning  evokes  different  things  to  different  people.  That  is,  it 
may  be  perceived  as  a mix  of  pedagogies,  Web-based  technologies,  as  well  as  the  combination  of  any  form  of 
instructional  technology  with  face-to-face  (FTF)  instructor-led  training.  This  is  probably  due  to  the  lack  of  a 
definite  conceptual  and  pedagogical  framework  for  what  blended  learning  serves  for  (Kerres  & De  Witt,  2003). 
There  is  a need  to  investigate  the  experiences  and  roles  of  instructors  in  the  entire  context  for  the 
conceptualization  of  blended  learning.  An  understanding  of  what  is  going  on  in  a blended  learning  environment 
and  the  perceptions  of  the  people  affected  from  it  can  contribute  to  a much-needed  knowledge  for  successful 
learning.  It  is  the  aim  of  the  researchers  to  create  an  understanding  of  the  design,  development  and 
implementation  processes  in  a blended  learning  environment  from  the  instructor’s  perspectives  and 
experiences.  This  study  is  a preliminary  case  to  understand  the  context  of  a sample  blended  environment  from 
instructor  experiences. 


Method 


Background  of  the  Study 

In  this  study,  an  undergraduate  course,  CEIT  209:  Computer  Aided  Instruction,  was  designed  in  a 
blended  learning  format  at  fall  semester  (September-January)  in  2005.  The  course  is  given  to  sophomores  in 
Computer  Education  and  Instructional  Technology  Department  (CEIT)  in  Middle  East  Technical  University, 
Turkey.  It  is  a must  course  and  was  given  in  FTF  format  until  2005  fall  semester. 

Before  the  design  of  the  course,  the  researcher  analyzed  students,  content,  and  context  of  the  course. 
This  process  began  after  the  fall  semester  in  2004,  in  which  the  course  was  offered  in  a FTF  environment.  At 
the  end  of  fall  semester  in  2004,  semi- structured  interviews  were  done  with  five  students  and  the  course 
instructor  to  gather  data  for  their  experiences  and  perceptions  on  FTF  environment.  The  analysis  of  these  data 
was  done  with  a content  analysis  process  as  Miles  and  Huberman  (1994)  described.  The  results  of  these 
interviews  showed  the  picture  of  the  course  context  and  provided  data  for  the  redesign  of  the  course  in  blended 
format.  What  is  meant  as  blended  learning  in  the  context  of  this  study  is  a mix  of  FTF  environment  and  online 
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learning  environment.  The  infrastructure  and  base  of  the  online  environment  of  the  course  was  created  and 
developed  by  Ozden  (2002),  who  was  using  this  system  in  his  undergraduate  and  graduate  classes  over  since 
1998.  It  was  revised  and  improved  by  taking  user  reactions  during  these  years.  This  environment  was  adapted 
to  the  needs  of  the  course. 

After  designing  and  developing  the  blended  learning  environment,  the  researcher  implemented  the 
course  as  the  instructor.  She  had  diaries  during  all  processes  in  order  to  stimulate  critical  reflection  of  her 
experiences.  In  the  diaries,  which  are  the  main  sources  of  data  in  this  study,  she  reflected  on  important 
processes  that  she  had  experienced.  These  diaries  were  analyzed  by  the  researcher  with  peer  debriefing  to 
confirm  the  credibility  of  the  information  which  she  had  interpreted. 

Design  of  the  Study 

This  research  is  conducted  as  a case  study.  As  Merriam  (1998)  indicates,  case  study  design  is  used  to 
understand  context-related  phenomenon  while  anchoring  in  real-life  situations.  The  case  study  is  an  appropriate 
method  of  inquiry  for  this  study  since  the  researcher  aims  to  understand  a particular  phenomenon  in  its  natural 
setting  in  an  in-depth  way.  The  data  were  gathered  from  documents  included  instructor  diaries,  learners’ 
portfolios  and  logs  in  Web  site,  and  the  other  course  related  data  like  web  site  content,  forum  discussions,  in- 
class  materials,  etc. 

The  first  author  of  this  research  was  the  instructor  of  this  course,  and  designed,  developed,  and 
implemented  the  course.  Being  a teaching  assistant  in  the  department,  the  researcher  is  familiar  with  the  socio- 
cultural context  of  the  department.  Having  the  instructor  role  also  provided  the  researcher  to  be  an  active 
participant  in  the  context  of  the  study.  This  gave  a direct  insight  into  a very  different  set  of  challenges.  That  is, 
it  allowed  an  in-depth  analysis  of  the  content  of  the  course,  student  engagement,  and  the  dynamics  of  the 
context  of  the  study.  As  Bogdan  and  Biklen  (1998)  noted,  since  it  is  difficult  to  get  away  from  who  you  are, 
and  what  you  believe;  the  study  is  affected  by  researcher’s  biases.  The  researcher  tried  to  avoid  this  threat  by 
expert  checking,  peer  observations,  and  peer  debriefing  methods.  Two  experts  from  departments  of  CEIT  and 
Educational  Sciences  guided  the  processes  during  the  design,  development  and  implementation  of  the  course. 
These  experts  provided  feedback  on  the  outcomes  of  document  analyses  and  analyses  of  field  notes  of  the 
instructor.  Two  peers,  who  were  doctorate  students  in  the  same  department  and  familiar  to  blended  learning 
environments  observed  the  courses  in  FTF  environment,  and  also  tracked  and  examined  the  course  Web  site. 
The  researcher  asked  these  peers  about  the  findings  and  took  feedback  on  the  analyses  of  field  notes  and 
documents. 


Results  and  Discussion 

The  instructor  had  different  experiences  in  the  two  environments  of  the  blended  environment  while 
designing,  developing  and  implementing  the  course:  FTF  environment  and  online  environment.  Even  if  they 
were  mixed  in  the  course,  each  had  its  own  characteristics  and  challenged  the  instructor  from  different  aspects. 

The  Design  of  the  Blended  Course 

The  core  of  the  design  of  the  blended  environment  was  based  on  creating  a flexible  and  rich  learning 
environment,  which  would  support  interaction  and  allow  student  collaboration  and  cooperation.  Rather  than 
presenting  the  information  only,  a two-way  communication  between  learners  and  the  instructor  was  also  aimed. 
The  results  of  the  interviews  with  students  and  instructor  of  the  course  in  Fall  2004  semester  indicated  that  the 
instructor  should  be  cautious  about  the  activities  design,  interaction,  learner  support,  and  online  course 
integration.  Therefore,  with  the  core  of  the  design  considerations  and  the  results  of  the  past  interviews,  the 
instructor  designed  the  course  as  in  blended  fonnat  while  taking  each  environment  into  consideration.  The  most 
important  challenge  that  the  researcher  faced  was  to  make  a balance  between  online  activities  and  FTF 
activities  to  make  them  go  hand  in  hand.  Other  than  this,  another  challenge  was  the  time  required  to  develop 
both  FTF  and  online  activities. 

FTF  Environment:  The  activities  were  designed  to  support  student  learning  flexibly  and  to  allow  active 
student  participation  by  incorporating  a bunch  of  variety.  Additionally  as  Arends  (1998)  suggested,  to  design  a 
rich  learning  environment,  designers  should  promote  self-regulated  learning.  The  activities  that  the  instructor 
designed  included  expert  seminars;  assignments  like  book  reviews  and  sample  software  evaluations;  student 
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presentations;  cooperative  learning  activities  in-class;  group  projects  out-of-class;  and  student  presentations  of 
the  projects. 

For  the  FTF  interactions  of  the  course,  the  instructor  aimed  to  create  a friendly  environment  which 
would  support  all  types  of  interaction  as  Moore  (1989)  identified:  student-content,  student-teacher,  and  student- 
student.  The  group  activities,  assignments,  and  expert  seminars  were  designed  to  serve  for  student-content 
interaction.  Student  presentations  and  learner-centered  approaches  to  classroom  teaching  of  the  instructor  were 
designed  to  promote  student-teacher  interaction,  while  cooperative  learning  activities,  group  projects,  and 
student  presentations  were  designed  to  serve  for  student-student  interaction. 

Online  Environment:  For  the  design  of  the  online  environment,  the  instructor  planned  to  use  a local  Learning 
Management  System.  As  the  theoretical  framework  of  the  study  embraced  a student-centered  approach  by 
encouraging  active  learning,  the  researcher  aimed  to  incorporate  the  following  considerations  as  Levin  et  al. 
(2001)  proposed: 

1.  Rich  student-student  interactions  through  discussions  in  forums,  instant  messaging  environment  and 
group  work  assignments, 

2.  Adequate  and  timely  feedback  from  instructors  through  messages  in  forums,  e-mail  posts. 

3.  Relevant  and  challenging  assignments  with  guidelines  and  relevant  resource  links, 

4.  Flexibility  in  teaching  and  learning,  by  allowing  individual  pace  in  the  completion  of  any  given 
assignment, 

5.  Coordinated  learning  environments  by  scheduling  tasks  at  the  beginning  of  the  term. 

Other  than  these,  according  to  Falvo  and  Solloway  (2004)  course  designers  need  to  address  the  socio- 
cultural context  of  the  online  learning  environment,  as  well  as  the  content  or  subject  matter,  which  is  being 
addressed  in  their  courses.  In  order  to  support  socio-cultural  context,  it  is  important  to  integrate  the  critical 
elements  into  the  learning  environment:  collaboration  and  conversation.  Collaboration  was  supported  in  both 
environments.  However,  conversation  was  only  in  FTF  environment  since  the  researcher  did  not  see  it  as  a 
necessity  for  online  environment. 

The  Development  of  the  Blended  Course 

The  development  process  began  together  with  the  design  phase.  During  the  development  of  the  course, 
content  development,  resource  allocation,  and  creation  of  the  course  web  site  both  in  technical  and  instructional 
means  challenged  the  researcher.  Findings  resources  from  the  Internet  and  print  and  audiovisual  sources  took  a 
lot  of  time.  She  could  not  even  finish  this  process  before  the  course  began.  Instead,  she  continued  this  process 
throughout  the  term  in  weekly  periods. 

FTF  Environment:  Developing  FTF  activities  challenged  the  instructor  in  terms  of  content  and  resource 
allocation.  However,  it  did  not  require  much  time.  After  each  week’s  course,  several  iterations  were  done  to 
improve  the  activities  and  relate  them  to  students’  needs.  Since  the  course  was  an  introductory  course  and 
students  did  not  have  a conceptual  background  on  the  topic,  the  researcher  needed  to  develop  more  detailed 
guidelines  for  assignments,  and  allocate  more  fundamental  resources. 

Online  Environment:  The  online  environment  had  a non-linear,  and  dynamic  structure.  At  the  beginning  of 
the  term,  the  researcher  conducted  a usability  study  with  a user  test  with  seven  students  that  are  registered  the 
course  and  expert  evaluations  with  Nielsen’s  (2005)  ten  heuristics,  and  redesigned  the  course  accordingly.  The 
students  could  have  access  to  web  site  with  user-ids  and  passwords  due  to  security  and  personalization  reasons. 
The  students  were  monitored  with  logs.  The  web  site  included  the  following  main  categories: 

• Course  materials 

o Course  Information 
o Content 
o Resources  links 

• Assignment  links 

o Book  review 
o Software  evaluation 
o Homework 

• Announcements 
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• Forum 

o Student  and  instructor  discussions 
o Reflections  on  projects 
o Reflections  on  assignments 
o Student  diaries 

The  Implementation  of  the  Blended  Course 

In  this  part,  the  findings  are  presented  together  for  the  two  environments.  The  reason  behind  this  is 
that  when  the  course  began,  the  experiences  for  two  environments  affected  each  other.  Throughout  the 
implementation  of  the  course,  the  researcher  came  across  to  challenges  that  she  could  not  imagine.  Classroom 
management,  time  management,  interaction  and  communication,  and  guidance  were  the  most  important  of 
them.  For  example,  several  students  not  volunteering  the  FTF  activities  at  all  or  not  participating  the  FTF 
sessions  were  actively  responding  to  questions  in  forums.  The  students  that  actively  participated  in  FTF  course 
were  not  interested  in  forum  or  any  interactive  communication  tool.  Since  the  online  environment  lacks  mimics 
and  gestures,  the  instructor  had  difficulty  to  manage  those  students  and  communicate  with  them.  This  can  be  a 
challenge  for  the  instructor  since  he/she  is  the  guide  of  that  environment.  Some  students  stated  that  since  they 
know  the  instructor  in  FTF  environment,  they  could  understand  what  she  meant  by  her  writings  in  online 
environment.  Flowever,  this  cannot  be  true  for  the  instructor,  since  she  may  not  know  how  to  behave  a student 
in  online  environment  if  the  student  does  not  participate  in  FTF  courses.  Another  important  thing  was  time 
management  for  the  instructor.  In  addition  to  weekly  FTF  meetings  and  office  hours,  the  instructor  needed  to 
spend  most  of  her  time  reading  postings,  handling  them,  tracking  students,  and  updating  information  in  the  site. 
She  tried  to  be  available  during  whole  week  in  front  of  a computer  for  the  students.  In  addition,  since  the 
number  of  students  was  too  much  for  online  discussions  (N=51),  the  discussions  could  be  done  within  groups. 

Beside  the  afore-mentioned  challenges,  allowing  a flexible  environment  and  incorporating  practical 
activities  increased  the  motivation  of  students,  which  satisfied  the  instructor.  The  instructor  believed  that  with 
the  strengths  of  the  two  environments,  she  provided  a flexible  and  self-regulated  learning  environment.  This 
enabled  students  to  communicate  and  interact  with  each  other  and  instructor;  and  cooperate  and  collaborate  for 
creating  real  world  projects  both  in  class  and  out-of-class  environments.  She  could  also  coordinate  them  and 
provided  feedback  throughout  the  week,  and  allowed  self-pacing  by  providing  all  materials  including 
presentation  documents  in  both  environments. 
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Figure  1.  Instructor  experiences  in  the  blended  learning  course  development. 


The  results  of  the  study  showed  that  when  blended  learning  is  selected  for  the  course  format,  the 
instructor  needed  to  deal  and  pursue  the  harmony  of  two  environments  as  well  as  dealing  with  their  features 
within  their  own  contexts  as  stated  in  the  literature.  Blending  FTF  and  online  environments  require  thoughtful 
design  and  implementation  since  they  pursue  different  student  and  time  management,  communication, 
interaction,  and  guidance  features.  The  results  may  help  researchers  and  practitioners  to  be  informed  of  the 
factors,  strategies  and  challenges  that  need  to  be  taken  into  account  when  utilizing  a blended  learning 
environment 
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The  Perceptions  and  Experiences  of  Preservice  Students  in  a Blended 

Learning  Environment 
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Abstract:  The  purpose  of  this  research  is  to  examine  and  describe  the  perceptions 
and  experiences  of  preservice  students  in  a blended  learning  environment.  In  this 
study,  data  collection,  data  analysis  and  inference  procedures  were  handled  through 
qualitative  method.  The  findings  indicated  that  the  students  perceive  blended 
environment  very  helpful  due  to  its  timeless  communication,  resource  and  material 
supply,  and  interaction  opportunities  as  well  as  social  negotiations  together. 

Key  Words:  Blended  Learning,  Online  Course,  Interactive  Systems,  Technology  Integration 


Introduction 

The  advent  and  exponential  growth  of  the  Internet  had  led  to  drastic  changes  in  the  creation  of  learning 
environments  in  educational  settings  (Chuang  & Wang,  2003;  Rosenberg,  2000).  However,  face-to-face  (FTF) 
communication  is  still  the  richest  medium  with  its  potential  for  intensive  interpersonal  communication  and  for 
building  social  relations.  In  relation  to  these,  blended  learning  has  piqued  the  interest  of  researchers  looking  for 
the  most  effective  options  for  learning  in  recent  years.  Singh  (2003)  argues  that  with  its  potential  to  provide 
multiple  medium  rather  than  single  medium,  blended  learning  becomes  a better  alternative  for  educators. 

Blended  learning  remains  a relatively  new  concept  in  the  field  of  Instructional  Technology  despite  its 
growing  popularity.  According  to  Garrison  and  Kanuka  (2004),  researchers  should  explore  the  impact  of  blended 
learning  in  achieving  meaningful  learning  experiences.  This  case  study  aimed  at  examining  and  describing 
students’  experiences  in  and  their  perceptions  on  the  blended  environment  in  an  undergraduate  course.  It  was  a 
preliminary  work  to  understand  the  context  of  a sample  blended  environment.  The  information  provided  by  the 
outcome  of  this  study  can  provide  insight  into  the  dynamics  of  design  and  implementation  of  a blended  learning 
environment  through  the  eyes  of  the  students. 


Method 

A qualitative  case  study  guided  the  study  since  the  nature  of  the  research  required  a detailed  context- 
related  description  of  the  phenomenon  (Yin,  1984).  According  to  Merriam  (1998),  a case  study  design  is  used  in 
order  to  understand  context-related  phenomenon  while  anchoring  it  in  real-life  situations.  The  case  study  was  an 
appropriate  method  of  inquiry  for  this  study,  since  the  researcher’s  goal  was  to  understand  a particular 
phenomenon  in  its  natural  setting  in  an  in-depth  way. 

The  data  were  collected  through  document  analyses  and  semi-structured  interviews  in  CEIT  209  course. 
This  course  is  offered  in  fall  semesters  at  the  Department  of  Computer  Education  and  Instructional  Technology 
(CEIT)  in  Middle  East  Technical  University,  Turkey.  It  is  a basic  introductory  course  about  Computer  Aided 
Instruction.  The  format  of  CEIT  209  course  was  a FTF  instruction  until  2005  fall  semester.  At  fall  2005,  a total 
of  fifty  one  students  took  this  course  in  blended  fonnat.  What  is  meant  as  blended  here  is  a mix  of  FTF  and 
online  environments.  The  infrastructure  and  base  of  the  online  environment  of  the  course  was  created  and 
developed  by  Ozden  (2002),  who  was  using  this  system  in  his  undergraduate  and  graduate  classes  over  since 
1998.  It  was  revised  and  improved  by  taking  user  reactions  during  these  years.  This  environment  was  adapted  to 
the  needs  of  the  course.  The  documents  were  gathered  throughout  the  semester  (October- January)  from  all 
students’  data.  Interviews  were  conducted  with  ten  students  (20  %). 

The  documents  , forum  messages  of  the  students,  were  organized  as  in  the  form  of  student  diaries.  The 
students  wrote  their  experiences  and  perceptions  of  the  use  of  Web  site  as  a support  environment  for  FTF  course 
and  experiences  about  the  format  of  the  course.  The  data  from  these  documents  were  analyzed  according  to  the 
data  analysis  procedure  that  Bogdan  and  Biklen  (1998)  propose:  emerging  themes  were  identified  by  reading 
data,  organizing  them,  breaking  them  into  manageable  units,  synthesizing  them,  searching  for  patterns, 
discovering  what  is  important,  and  deciding  what  to  present  and  tell. 

Two  semi-structured  interviews  were  conducted  with  ten  students  at  the  beginning  of  the  term  (October 
2005),  and  with  fifteen  students  at  the  end  of  the  tenn  (January  2006).  For  all  interviews,  the  researcher 
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developed  interview  guides  and  asked  one  expert  and  three  peers  to  evaluate  it.  As  they  were  semi-structured 
interviews,  the  researcher  found  opportunities  to  ask  further  questions  appropriate  to  the  flow  of  the  interviews. 
After  completing  data  gathering  part,  data  were  transcribed,  coded  and  analyzed  with  the  content  analysis 
process  as  Miles  and  Huberman  (1994)  described.  During  data  analysis,  the  researchers  asked  two  doctorate 
students  outside  of  the  study  for  feedback  and  maintain  intercoder  reliability,  who  were  familiar  with  the  nature 
of  the  study  and  have  had  experience  on  interviewing. 

Findings  and  Discussion 

In  the  first  interviews,  researchers  collected  data  about  the  students’  perceptions  on  having  the  CEIT 
209  course  in  blended  format.  The  results  revealed  that  the  students  were  eager  for  the  Web  site  as  (a)  support 
for  the  FTF  course.  All  of  the  interviewees  stated  that  it  would  be  very  helpful  and  encouraging  for  them  to 
communicate  and  interact  with  peers  and  with  the  instructor  throughout  the  semester,  not  only  in  the  class  hours. 
They  also  mentioned  about  the  opportunity  to  have  access  to  the  course  materials,  which  would  motivate  them  to 
study  on  course  content  out  of  the  class.  All  these  positive  comments  showed  that  the  students  were  enthusiastic 
about  taking  the  course  in  blended  format. 

Throughout  the  course,  the  forum  messages  showed  that  the  students  could  be  able  to  discuss  the  topics 
that  they  could  not  discuss  in  the  lesson  due  to  time  constraints  or  other  reasons,  and  they  could  be  able  to 
communicate  with  the  instructor  and  peers  whenever  they  needed.  In  addition  to  these,  it  was  interesting  that  the 
students  that  were  not  attending  or  not  participating  at  all  in  the  FTF  environment  could  participate  to  the  course 
in  Web  environment.  That  is,  they  could  make  use  of  the  benefits  of  Web  and  FTF  environment  together  in  the 
course. 

The  second  interview  was  about  the  experiences  of  the  students  and  their  perceptions  on  the  blended 
learning  environment.  The  results  of  the  interviews  indicated  that  the  students  could  benefit  from  both  FTF  and 
online  environment.  The  major  benefits  mentioned  by  the  interviewees  were  the  flexibility  in  time  to  study 
course,  wide  allocation  of  course  resources  and  materials,  interaction  opportunities  (both  with  the  peers  and  the 
instructor),  instant  feedback  from  the  course  instructor  and  class  members  in  negotiation,  opportunity  to  discuss 
the  topics  in  online  environment  that  they  could  not  discuss  in  the  class,  and  opportunity  to  discuss  topics  with  a 
FTF  interaction  and  to  get  immediate  feedback. 

For  the  barriers  that  they  had,  the  students  mentioned  about  some  technical  problems  with  the  online 
environment.  They  could  not  sometimes  connect  to  the  forum  environment  due  to  password  problems  at  the 
beginning  of  the  term,  but  soon  it  was  solved.  Two  interviewees  also  complained  about  the  crowd  of  the  people 
taking  the  course  together.  They  expressed  that  they  were  sometimes  getting  difficulty  in  concentrating  FTF 
discussions,  but  they  could  continue  these  discussions  in  forum  environment  comfortably.  One  interviewee  also 
stressed  the  importance  of  the  social  negotiations  within  FTF  environment.  He  stated  that  the  opportunity  of 
having  a fifteen  minutes  FTF  discussion  was  worth  hours  of  the  forum  discussions  for  him.  However,  he  also 
noted  that  accessing  course  materials  and  follow  announcements  online  was  so  effective  for  him  that  he  would 
not  prefer  just  the  FTF  environment. 

Table  1.  Student  perceptions  of  blended  learning 


Benefits 

Barriers 

Suggestions 

* Timeless  communication-interaction 

* Increasing  workload 

* More  guidelines  for  the 
process  of  FTF  and  online 
environments 

* Wide  allocation  of  course  resources 
and  materials 

* T echnical  problems  with 
online  environment 

* More  discussions  in  online 
environment  within  group 

* Social  interaction  both  in  and  out  of 
class 

* Flexibility  in  time  to  study  course 

* Instant  feedback  from  the  course 
instructor  and  class  members  in 
negotiation 

* Opportunity  to  discuss  the  topics  in 
online  environment  that  they  could  not 
discuss  in  the  class 

* Opportunity  to  discuss  topics  with  a 
FTF  interaction 

* More  student  participation 
to  the  design  of  the  course 
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In  line  with  what  literature  says  about  blended  learning,  this  pilot  study  showed  that  blended  learning 
environment  can  offer  a flexible  environment  to  the  diverse  needs  of  students  in  higher  education.  All  the 
features  may  not  catch  students’  interests  or  motivate  them,  but  providing  multiple  opportunities  to  students  can 
increase  the  flexibility  of  the  learning  environment.  It  can  also  be  concluded  that  even  if  it  takes  time  for  students 
to  deal  with  the  courses,  having  access  to  course  materials,  resources  and  opportunity  to  interact  with  instructor 
and  peers  support  student  learning  in  a blended  environment. 
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Abstract 

This  study  investigates  faculty  members’,  preservice  teachers’,  and  K-12  teachers’  opinions 
about  effectiveness  of  ICT  related  courses  in  preservice  teacher  education  programs  in  Turkey. 
The  design  of  the  study  is  cross-sectional  survey  method.  The  data  were  collected  from  1 1 1 
faculty  members,  1330  preservice  teachers,  and  1429  K-12  teachers  with  three  questionnaires. 
The  findings  indicated  that  majority  of  participants  agreed  for  the  courses  providing  to  the 
preservice  teachers  necessary  knowledge  and  skills  in  using  and  integrate  ICT. 


Key  Words:  Evaluation  of  ICT  Related  Courses,  Effectiveness  of  ICT  Related  Courses,  ICT 
Related  Courses,  Technology  Integration  Courses,  Preservice  Teacher  Education,  and  PT3 

Introduction 

The  rapid  developments  in  information  and  communication  technology  (ICT)  affected  the  educational 
paradigm  in  the  world.  Teacher  education  programs  across  the  countries  have  been  challenged  to  respond  to 
these  developments.  Parallel  to  the  international  practices  in  reforming  preservice  teacher  education  for  the 
new  millennium,  in  1998,  the  Higher  Education  Council  in  Turkey  reconstructed  Faculties  of  Education’ 
curriculum  to  train  teacher  candidates  with  abilities  and  skills  to  use  ICT  effectively  in  their  subject  areas. 
According  to  the  new  curricula,  it  became  compulsory  for  all  faculties  of  education  students  to  take 
“Computer”  and  “Instructional  Technologies  and  Material  Development  (ITMD)”  courses  to  fulfill  the 
requirements  for  teaching  credential  (HEC,  1998). 

The  new,  technology  integrated  preservice  teacher  education  curriculum  has  been  implemented 
since  1998.  However,  there  is  no  evidence  on  the  effectiveness  of  the  new  curriculum,  and  it  is  not  clear 
that  if  the  new  curriculum  meets  the  required  needs  in  ICT  training  of  prospective  teachers.  Therefore, 
there  is  a need  for  research  studies  on  the  effectiveness  of  the  new  ICT  related  courses.  Hence,  the  purpose 
of  this  study  is  to  investigate  the  effectiveness  of  ICT  related  courses  in  Turkey’s  preservice  teacher 
education  programs  in  regard  to  faculty  members,  preservice  teachers,  and  K-12  teachers’  perceptions. 
Consequently,  this  study  addressed  the  following  research  questions: 

(1)  How  do  faculty  members,  preservice  teachers  and  K-12  teachers  perceive  the  effectiveness  of 
“Computer”  course  in  terms  of  ICT  integration  into  preservice  teacher  education? 

(2)  How  do  faculty  members,  preservice  teachers  and  K-12  teachers  perceive  the  effectiveness  of 
“ITMD”  course  in  terms  of  ICT  integration  into  preservice  teacher  education? 

Method 

The  design  of  the  study  is  cross-sectional  survey  method.  In  order  to  collect  relevant  data  on  the 
stakeholders’  perceptions  of  effectiveness  of  ICT  related  courses  in  preservice  teacher  education  programs,  three 
questionnaires  were  used.  The  first  questionnaire  (Ql)  was  developed  by  the  researchers  to  gather  data  from 
the  faculty  members  who  give  ICT  related  courses  in  preservice  teacher  education  programs,  and  consisted 
of  24  items.  The  second  questionnaire  (Q2)  was  used  to  collect  data  from  preservice  teachers  who  had 
taken  ICT  related  courses  before  spring  semester  of  2004-2005,  and  consisted  of  42  items.  It  was  developed 
originally  by  Tinmaz  (2004),  and  was  adapted  for  this  study.  The  last  questionnaire  was  developed  by  the 
researchers  to  gather  data  from  the  K-12  teachers  who  had  taken  ICT  related  courses  in  their  undergrad,  and 
consisted  of  16-items.  Each  questionnaire  was  examined  by  the  experts,  checked  by  a Turkish  Language 
expert,  and  piloted. 

The  data  were  collected  from  faculty  members  (n=l  1 1)  in  18  universities,  preservice  teachers 
(n=1330)  in  the  same  18  universities,  and  K-12  teachers  (n=1429)  in  92  schools  and  35  provinces  within  12 
regions  of  Turkey  with  a representative  and  convenience  sampling  method,  who  would  provide  valuable 
information  about  the  effectiveness  of  ICT  related  courses. 

The  data  gathered  through  questionnaire  were  analyzed  by  descriptive  statistics,  and  frequencies, 
means,  percentages  and  standard  deviations  were  calculated.  In  this  paper,  only  some  parts  of  the  data 
gathered  from  the  questionnaires  were  reported. 
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Results 

This  study  looks  at  perceptions  of  the  faculty  members,  prospective  teachers  and  K-12  teachers  about 
the  effectiveness  of  ICT  related  courses  in  teacher  education  programs.  There  were  two  ICT  related  courses,  so 
there  are  two  major  categories  in  this  theme.  The  first  category  is  “Computer”  and  second  is  “ITMD”.  The  data 
were  collected  from  faculty  members  through  surveys  consisting  of  3 -point  quantitative  scales  ( 1 indicating 
“Disagree”,  2 indicating  “Neutral”,  3 indicating  “Agree”).  The  data  from  prospective  and  K-12  teachers  were 
collected  with  a five  point  Likert-type  scale  (5  indicating  “Strongly  Agree”,  4 indicating  “Agree”,  3 indicating 
“Neutral”,  2 indicating  “Disagree”,  and  1 indicating  “Strongly  Disagree”).  The  means  and  standard  deviations  of 
the  quantitative  scales  for  all  participants  are  detailed  in  Table  1. 

Table  1:  Means  and  Standard  Deviations  of  Perceived  Effectiveness  of  ICT  Related  Courses 


“Computer” 

M 

Course 

SD 

“ITMD” 

M 

Course 

SD 

Faculty  Members  (on  a 3 -point  scale) 

2.60 

.55 

2.45 

.56 

Prospective  Teachers  (on  a 5-point  Likert  scale) 

3.59 

1.30 

3.85 

1.13 

K-12  Teachers  (on  a 5-point  Likert  scale) 

3.02 

1.43 

3.08 

1.33 

K-12  teachers  who  started  TES  in  1998  or  later 

3.55 

1.30 

3.58 

1.18 

K-12  Teachers  (on  a 5-point  scale) 

2.94 

1.32 

2.98 

1.36 

K-12  teachers  who  started  TES  in  1998  or  later 

3.46 

1.13 

3.56 

1.10 

Computer  Courses: 

In  this  section,  perceived  effectiveness  of  “Computer”  courses  of  faculty  members,  prospective 
teachers,  and  K-12  teachers  were  examined  in  teacher  education  programs.  The  following  categories  have  been 
created  for  this  purpose. 

Perceptions  of  Faculty  Members  about  Computer  Courses: 

Faculty  members  were  asked  about  the  effectiveness  of  the  course  titled  “Computer”  in  tenns  of  ICT 
integration  into  preservice  teacher  education  programs.  They  showed  a high  degree  of  overall  contentment  (M= 
2.60)  with  their  ICT  related  courses  and  felt  well  prepared  for  professional  life  (see  Table  1). 

When  asked  to  how  improve  the  course  retrospectively,  the  findings  show  that,  “in  whole  the  course 
should  be  offered  in  a computer  laboratory  based  on  applications”  (F=13),  was  ranked  first  and  “the  course 
content  should  be  redesigned  to  acquire  more  benefit  from  ICT  based  on  today  needs”  (F=  11)  is  second 
according  to  the  course  instructors  (see  Table  2). 

Table  2:  Ways  to  Improve  the  “Computer”  Course  According  to  Faculty  Members 


F 

% 

In  whole  the  course  should  be  offered  in  a computer  laboratory  based  on 
applications 

The  course  content  should  be  redesigned  to  acquire  more  benefits  from 
ICT  based  on  today  needs 

13 

11 

32 

27 

Appropriate  inservice  training  should  be  provided  to  the  faculty 
members  who  offer  the  course 

10 

24 

More  hardware  and  the  other  equipments  should  be  allocated  to  the  course 

7 

17 

TOTAL 

41 

100 

Perceptions  of  Prospective  Teachers  about  Computer  Courses: 

The  prospective  teachers  were  asked  about  the  effectiveness  of  the  course  titled  “Computer”  in  tenns  of 
ICT  integration  into  preservice  teacher  education  programs.  They  showed  a degree  of  overall  contentment  with 
their  “Computer”  course  and  felt  well  prepared  for  professional  life  (M=  3.59;  see  Table  1). 

As  shown  in  Table  3,  prospective  teachers  were  also  asked  some  indirect  questions  about  the 
effectiveness  of  both  ICT  related  courses.  The  findings  show  the  overall  mean  of  this  group  of  items  were  at 
the  agree  level,  which  means  the  course  was  beneficial  and  effective.  However,  “I  was  taught  to  (how  to)  use 
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technology  in  learning-teaching  environments  by  my  instructors  in  undergraduate  years”  (M=3.26)  and  “I  was 
taught  about  the  effects  of  technology  use  in  society  by  my  instructors  in  undergraduate  years”  (M=3.12)  were  at 
the  undecided  (neutral)  level. 

Table  3:  Means  and  Standard  Deviations  of  Indirect  Questions  to  Perceive  the  Effectiveness  of  Both  (ICT 
Related)  Courses 


M 

SD 

The  technology  courses  in  my  undergraduate  years  helped  me  to  change  my 
attitude  toward  technology  positively 

3.75 

1.11 

I am  capable  of  providing  technology-based  instruction  with  the  help  of 
technology-based  courses  that  I took  in  my  undergraduate  years 

3.53 

1.12 

I am  a prospective  teachers  that  is  capable  of  today’s  technology  criteria 

3.42 

1.17 

I was  taught  to  (how  to)  use  technology  in  learning-teaching  environments 

3.26 

1.22 

by  my  instructors  in  undergraduate  years 

I was  taught  about  the  effects  of  technology  use  in  society  by  my  instructors 

3.12 

1.20 

in  undergraduate  years 

Overall  mean 

3.41 

Perceptions  of  K-12  Teachers  about  Computer  Courses: 

The  K-12  teachers  were  asked  to  rate  the  effectiveness  of  the  course  titled  “Computer”  in  terms  of  ICT 
integration  into  preservice  teacher  education  programs.  They  rated  their  levels  of  agreement  on  the  effectiveness 
of  the  course  with  the  statements  by  using  two  themes. 

The  first  theme  examined  in  this  study  was  the  way  of  acquiring  ICT  competencies  by  K-12  teachers. 
Means  and  total  percentages  of  teachers  who  marked  “agree  or  strongly  agree”  are  provided  in  Table  1 . As  is 
presented,  “Computer”  course  taken  at  their  undergraduate  education  is  one  factor  contributing  to  their 
acquisition  of  competency  (47.3%,  M=3.02).  Other  factors  contributing  to  acquiring  competency  in  ICT  for  K- 
12  teachers  are  at  a “neutral”  level  (ranging  from  M=2.61  to  M=3.40).  When  the  researcher  focused  only  on 
the  K-12  teachers  with  a starting  year  to  the  TES  as  1998  and  above,  the  mean  was  calculated  at  “Agree” 
levels  (M=3.55). 

The  second  theme  examined  in  this  study  was  the  effectiveness  of  “Computer”  course  in  their 
undergraduate  education  which  increased  the  knowledge  and  skills  of  ICT  integration  into  their  teaching 
profession.  K-12  teachers  rated  their  level  of  agreement  related  with  each  contributing  factor  by  using  a five- 
point  Likert-type  scale  (5  indicating  “Strongly  Agree”,  4 indicating  “Agree”,  3 indicating  “Disagree”,  2 
indicating  “Strongly  Disagree”,  and  1 indicating  “They  do  not  have  idea  / had  not  taken  “Computer”  courses 
during  their  undergraduate  study”).  K-12  teachers  were  undecided  (neutral  level)  about  the  overall  contentment 
with  their  ICT  related  courses  (M=  2.94).  When  the  researcher  focused  only  on  the  K-12  teachers  with  a 
starting  year  to  the  TES  as  1998  and  above,  the  mean  was  calculated  at  “Agree”  levels  (M=3.46).  K-12 
teachers  showed  a degree  of  overall  contentment  with  their  “Computer”  course  and  felt  well  prepared  for 
professional  life. 

Instructional  Technologies  and  Material  Development  Courses: 

In  this  section,  perceived  effectiveness  of  the  “ITMD”  course  by  faculty  members,  prospective  teachers 
and  K-12  teachers  were  examined  in  teacher  education  programs.  The  following  categories  have  been  created  for 
this  purpose. 

Perceptions  of  Faculty  Members  about  ITMD  Courses: 

Faculty  members  were  asked  about  the  effectiveness  of  the  course  titled  “ITMD”  in  terms  of  ICT 
integration  into  preservice  teacher  education  programs.  They  showed  a high  degree  of  overall  contentment  with 
their  ICT  related  courses  and  felt  well  prepared  for  professional  life  (M=  2.45;  see  Table  1). 

They  were  also  asked  about  how  to  improve  the  course  retrospectively.  Findings  show  that,  “the  course 
benefits  (acquirement)  should  be  implemented  in  the  method  courses”  (F=26),  is  the  ranked  highest  and  “more 
electronic  classroom  and  computer  laboratories  should  be  allocated  to  the  course”  (F=20)  is  the  second 
choice  according  to  the  course  instructors. 

Table  4:  Ways  to  improve  the  “ITMD”  Course  According  to  Faculty  Members 

F % 

The  course  benefits  (acquirement)  should  be  implemented  in  the  ^ 34 

method  courses 
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More  electronic  classroom  and  computer  laboratories  should  be 
allocated  to  the  course 

Appropriate  inservice  training  should  be  given  to  the  faculty 
members,  who  offer  the  course 

The  course  content  should  be  redesigned  to  acquire  more 
benefit  from  ICT  based  on  today  needs 

TOTAL 


20 

26 

19 

25 

12 

15 

77 

100 

Perceptions  of  Prospective  Teachers  about  ITMD  Courses: 

The  prospective  teachers  were  asked  to  rate  the  effectiveness  of  the  course  titled  “ITMD”  in  terms  of 
ICT  integration  into  preservice  teacher  education  programs.  Prospective  teachers  showed  a degree  of  overall 
contentment  (M=  3.85)  with  their  “ITMD”  course  and  felt  well  prepared  for  professional  life  (see  Table  1). 

Perceptions  of  K-12  Teachers  about  ITMD  Courses: 

The  K-12  teachers  were  asked  about  the  effectiveness  of  the  course  titled  “ITMD”  in  terms  of  ICT 
integration  into  preservice  teacher  education  programs.  They  rated  their  levels  of  agreement  on  the  effectiveness 
of  the  course  with  the  statements  by  using  two  themes. 

The  first  theme  examined  in  this  study  was  the  way  of  acquiring  ICT  competencies  by  K-12  teachers. 
Means  of  teachers  who  marked  “agree  or  strongly  agree”  are  provided  in  Table  1.  The  results  indicate  that 
“ITMD”  taken  during  undergraduate  education  (49.4%,  M=3.08),  is  the  highest  factor  that  contributed  to 
acquiring  competency  in  ICT  of  K-12  teachers;  the  other  factors  are  at  “neutral”  levels  (ranging  from  M=2.61 
to  M=3.40).  When  the  researcher  focused  only  on  the  K-12  teachers  with  starting  year  to  the  TES  as  1998 
and  above,  the  mean  was  calculated  at  “Agree”  levels  (M=3.58). 

The  second  theme  examined  in  this  study  was  the  effectiveness  of  the  “ITMD”  course  during  the  K-12 
teachers’  undergraduate  education  if  the  course  increased  the  knowledge  and  skills  of  ICT  integration  into  their 
teaching  profession.  They  rated  their  level  of  agreement  related  with  each  contributing  factor  by  using  a five- 
point  Likert-type  scale  (5  indicating  “Strongly  Agree”,  4 indicating  “Agree”,  3 indicating  “Disagree”,  2 
indicating  “Strongly  Disagree”,  and  1 indicating  “They  do  not  have  idea  / had  not  taken  “ITMD”  course  during 
their  undergraduate  study”).  K-12  teachers  were  undecided  (neutral)  about  the  overall  contentment  with  their 
ICT  related  courses  (M=  2.98).  When  the  researcher  focused  only  on  the  K-12  teachers  with  starting  year  to 
the  TES  as  1998  and  above,  the  mean  was  calculated  at  “Agree”  levels  (M=3.56).  Majority  of  them  showed 
a degree  of  overall  contentment  with  their  “ITMD”  course  and  felt  well  prepared  for  professional  life  (see  Table 
1). 

Implications 

Evidence  indicated  that  the  two  groups,  faculty  members  and  prospective  teachers,  perceived  ICT 
related  courses  beneficial  and  effective  in  ICT  integration  into  education  (see  Table  1).  K-12  teachers  showed  a 
degree  of  overall  unsure  (neutral)  with  their  ICT  related  courses  and  felt  well  prepared  for  professional  life. 
However,  majority  of  K-12  teachers  with  a starting  year  to  the  TES  as  1998  or  later,  indicated  agreement 
with  the  statements  “ICT  related  courses  are  effective  in  providing  prospective  teachers  with  the  necessary 
knowledge  and  skills  in  ICT”  (see  Table  1).  Thus,  the  present  research  results  parallel  the  literature  (Tinmaz, 
2004)  and  lend  support  to  HEC’s  (1998)  reform  in  terms  of  reconstructed  curriculum  to  train  prospective 
teachers  with  abilities  and  skills  to  use  ICT  effectively  in  their  subject  areas. 

The  majority  of  the  faculty  members  strongly  agreed  with  the  statements  “ICT  related  courses  are 
effective  in  providing  prospective  teachers  with  the  necessary  knowledge  and  skills  in  ICT”  than  the  prospective 
teachers  and  K-12  teachers. 

The  results  of  the  study  also  indicated  a majority  of  the  prospective  teachers  and  K-12  teachers 
perceived  the  “ITMD”  course  to  be  more  effective  than  the  “Computer”  course.  On  the  contrary,  faculty 
members  believe  that  the  “Computer”  course  was  more  effective  than  the  “ITMD”  course  to  meet  the 
required  needs  of  prospective  teachers  in  ICT  training  (see  Table  1). 

For  the  “ITMD”  course,  faculty  members  also  thought  it  could  improve  their  effectiveness  and 
efficiencies,  if  the  course  were  implemented  in  the  method  courses  and  more  electronic  classrooms  and 
computer  laboratories  should  be  allocated  to  the  course  (see  Table  4).  In  response  to  questionnaire  results, 
they  believed  the  “ITMD”  course  can  be  offered  in  the  second  year,  after  the  “Computer”  course.  Also,  course 
content  of  this  course  included  some  problem  and  project  based  learning  activities  in  order  to  teach  how 
prospective  teachers  integrate  ICT  into  their  fields  during  an  ill-structured  process  (SITE,  2002).  They  supposed 
the  majority  of  class  activities,  assignments,  and  projects  provided  more  computers  and  other  technological 
devices  rather  than  posters  and  3-D  models.  K-12  teachers  agreed  with  the  faculty  member  recommendation 
in  terms  of  exploring  ill-structured  problems  and  including  more  assignments  and  projects  related  to  the 
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ICT.  Some  of  the  faculty  members  and  K-12  teachers  have  a high  opinion  of  both  courses,  felt  instructors 
were  knowledgeable  about  instructional  technology,  and  that  instructors  should  be  chosen  from  the  field  of 
instructional  technology.  Toker  (2004)  stated  similar  opinions  in  his  study,  schools  of  education  should 
choose  teaching  staff  for  technology  courses  from  the  field  of  education. 
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Integration  of  ICT  into  Preservice  Teacher  Education:  Main  Barriers  and 

Possible  Solutions 

Yuksel  Goktas 
Zahide  Yildirim 
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Abstract 

The  puipose  of  this  research  is  to  investigate  the  main  barriers  of  and  the  possible  solutions 
for  integrating  ICT  into  Turkey’s  preservice  teacher  education  programs.  The  data  was 
collected  from  53  deans,  1 1 1 faculty  members,  and  1330  students  of  teacher  education 
schools  with  both  quantitative  and  qualitative  methods.  The  findings  indicated  that  majority 
of  all  stake  holders  believe  “lack  of  inservice  training  about  ICT”,  “lack  of  appropriate 
software  and  materials  for  instruction",  and  "lack  of  hardware”  are  significant  barriers  for 
integrating  ICT  into  preservice  teacher  education  programs.  Based  on  these  findings,  several 
possible  solutions  were  proposed  on  behalf  of  successful  ICT  integration. 


Key  Words:  ICT  Barriers,  ICT  Obstacles,  Technology  Integration  Problems,  ICT 
Integration  Problems,  Preservice  Teacher  Education,  ICT  Teacher  Training,  ICT,  PT3 

Introduction 

After  the  1980s,  information  and  communication  technologies  (ICT)  have  become  essential  for  teaching 
and  learning  environments  day  by  day.  These  drastic  changes  certainly  affect  today's  teacher  education  context. 
Initially,  technology  issues  quickly  moved  from  instituting  special  programs  for  preparing  individuals  to  become 
ICT  specialists  in  schools  and  then  to  infusing  ICT  into  all  aspects  of  an  educator's  preparation  (Glenn,  2002). 

With  the  above-mentioned  consequences,  many  action  plans  were  adopted  at  national  and 
international  levels,  as  well  as  investments  for  ICT  in  teacher  education.  Most  of  the  teacher  education 
programs  have  been  redesigning  their  curricula  in  order  for  preservice  teachers  to  become  competent  users 
of  new  technologies  when  they  become  teachers. 

In  spite  of  the  various  action  plans  for  ICT  integration  into  teacher  education,  many  barriers  for 
effective  integration  still  exist  in  practice.  To  facilitate  these  plans,  main  barriers  should  be  overcome 
absolutely.  While  some  of  the  teacher  education  programs  do  not  face  these  barriers,  some  others  have 
certain  problems  due  to  those  barriers.  Therefore,  current  situation  of  integrating  ICT  into  teacher  education 
is  still  a struggle  all  around  the  world.  In  their  study,  Ertmer  et  al.  (1999)  stated  for  the  struggle  of  using 
technology  effectively,  “it  may  be  important  to  look  at  what  they  have  (in  terms  of  beliefs  and  practices)  in 
addition  to  what  they  do  not  have  (in  terms  of  equipment)”.  Despite  these  two  reasons  fundamentally  form 
significant  obstacles,  several  barriers  can  be  identified  for  infusing  technology  into  teacher  education 
programs. 

According  to  Topp  et  al.  (1995)  and  Baron  and  Goldman  (1994  the  barriers  for  integrating  ICT  into 
preservice  teacher  education  are:  (1)  limited  availability  of  equipment;  (2)  lack  of  faculty  training;  (3)  no 
clear  expectation  that  faculty  will  incorporate  technology  in  academic  activities;  (4)  lack  of  funds;  (5)  lack 
of  time  to  develop  facility  in  using  equipment  and  software;  (6)  doubt  about  the  pedagogical  validity;  (7) 
lack  of  technical  support;  (8)  lack  of  appropriate  materials;  and  (9)  absence  of  clear  programmatic  goals  for 
the  teacher  education  program  as  a whole. 

Similar  to  the  above  barriers,  the  report  of  SchoolNetAfrica  (2004)  identified  the  these  barriers  in 
their  reports:  (1)  lack  of  ICT  experience  and  skills  among  teacher  educators;  (2)  lack  of  access  to 
technology  in  pre-service  training  institutions;  (3)  lack  of  access  to  ICT  training  content;  (4)  lack  of  access 
to  good  quality  research  (including  content  examples)  from  institutions  that  are  already  integrating  ICT  into 
pre-service  training. 

In  a similar  study,  Glazewski  et  al.  (2001)  also  studied  barriers.  While  findings  of  the  study  stated 
similar  barriers;  they  proposed  “preservice  teachers  did  not  perceive  potential  problems  such  as  preparation  time 
and  implementation  as  major  barriers  to  effective  integration”,  but  “lack  of  or  limited  access  to  computers  in 
schools”,  “not  enough  software  available  in  schools”,  and  “lack  of  knowledge  about  technology”.  On  the  other 
hand,  “the  faculty  indicated  that  lack  of  preparation  time  and  implementation  time  was  a major  reason  why 
technology  was  not  being  effectively  integrated  in  many  instructional  settings”  (p.4). 

Cuban’s  et  al.  (2001)  point  to  the  following  barriers  for  the  use  of  technology  more  innovatively: 

(1)  teachers  do  not  have  the  time  to  find  and  evaluate  software;  (2)  computer  and  software  training  was 
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seldom  offered  at  convenient  times;  (3)  most  of  the  available  training  was  too  generic  and  not  specific  to 
the  needs  of  the  teachers. 

According  to  Mumtaz  (2000),  there  are  three  interlocking  factors  that  affect  teachers’  use  of  ICT. 
First  is  the  school  as  an  institution  providing  insufficient  time  to  teachers  to  manage  and  familiarize 
themselves  with  ICT.  Second  is  limited  resources  within  schools,  which  is  a great  obstacle  to  the  take-up  of 
ICT.  Lack  of  computers  and  software  in  the  classroom  can  seriously  limit  what  teachers  are  able  to  do  with 
ICT.  And  the  last  is  teacher  factors  that  involved  beliefs  about  the  way  the  subject  should  be  taught  and 
skills  associated  with  competence  in  managing  classroom  activities  and  computer-handling  technical  skills, 
as  the  most  influential  in  teachers’  use  of  computers. 

In  the  light  of  above-mentioned  literature,  it  is  obvious  that  technology  integration  is  influenced  by 
many  barriers.  To  prepare  preservice  teachers  better  and  to  overcome  these  barriers  more  successful  in  order  for 
technology  integration,  possible  solutions  should  be  proposed. 

According  to  Scrimshaw  (2004),  there  are  two  factors,  which  enable  ICT  use  in  education.  One  of 
them  is  individual  factors  as  the  availability  of  high  quality  resources,  high  level  of  technical  support,  full 
access  to  software  and  hardware  at  all  times,  and  availability  of  good  quality  training.  And  second  is  school 
level  enabling  factors  which  include  staff  programme  of  ICT  training,  effective  timetabling  of  rooms  and 
equipment,  also  access  to  resources,  on-site  technical  support,  and  whole  school  policies  on  using  ICT 
across  curriculum. 

According  to  Sugar  (2002),  positive  attitude  of  teachers  toward  ICT  integration  into  the  classroom  is  the 
most  important  incentive.  By  changing  their  attitudes  toward  the  use  of  technology  in  schools,  teachers  could 
potentially  remove  several  obstacles  to  effective  ICT  integration. 

Following  items  might  be  also  solutions  to  overcome  the  significant  barriers:  adequate  equipment 
and  resources  in  the  literature  (Becker,  1994;  Fabry  & Higgs,  1997;  Hadley  & Sheingold,  1993;  OTA, 

1995;  Topp  et  al.,  1995);  allocating  specific  units  or  personnel  for  peer  support  and  to  help  reduce  of  the 
teacher  workload  (Japonite,  2001;  Pricewaterhousecoopers  2001;  Ronnkvist,  1998,  Becker,  1994;  OTA, 
1995);  staff  development  (OTA,  1995;  Willis,  1993);  preparation  of  technology  plans  for  implementing 
ICT  in  faculties  and  universities  (UNESCO,  2002). 

The  reasons  indicated  above  have  emerged  a new  interest  in  ICT  in  education  recently.  There  is  a 
need  for  more  studies  on  the  main  barriers  and  the  possible  solutions  for  integrating  ICT  into  preservice 
teacher  education  programs.  Therefore,  the  purpose  of  this  study  is  to  reveal  the  significant  barriers  faced 
and  the  possible  solutions  for  current  ICT  integration  problems  in  Turkey’s  preservice  teacher  education 
programs  according  to  deans,  faculty  members,  and  preservice  teachers.  Consequently,  this  study  addressed 
the  following  research  questions: 

1 . What  are  the  main  barriers  faced  in  integrating  ICT  into  preservice  teacher  education  programs 
according  to  deans,  faculty  members,  and  preservice  teachers? 

2.  What  are  the  possible  solutions  to  current  ICT  integration  problems  in  preservice  teacher 
education  programs  according  to  deans,  faculty  members,  and  preservice  teachers? 

Method 

Design  of  the  Study 

In  this  study,  the  researchers  used  both  quantitative  and  qualitative  research  approaches  within 
each  stage.  For  data  collection,  quantitative  measures  included  data  from  questionnaires  that  were  used  for 
deans  and  faculty  members.  Qualitative  measures  included  data  from  open-ended  questions  from 
questionnaires  and  semi-structured  interviews  that  were  used  for  deans,  faculty  members,  and  preservice 
teachers  to  report  their  perceptions  of  the  main  barriers  and  possible  solutions  for  integrating  ICT 
problems.  In  addition  to  data  collection,  both  qualitative  and  quantitative  methods  were  used  in  data 
analysis. 

Population  and  Sample 

Some  of  the  important  stakeholders  in  ICT  integration  into  preservice  teacher  education  programs 
are  deans,  faculty  members,  and  preservice  teachers.  In  this  study,  the  data  were  collected  from  these  three 
stakeholders,  who  would  provide  valuable  information  about  the  process  of  ICT  integration. 

Deans  of  Education  Faculties:  There  are  63  public  and  private  teacher  education  schools  that  train 
teachers  for  primary  and  secondary  education  in  Turkey.  In  order  to  collect  data  on  ICT  integration  into 
preservice  teacher  education  programs,  the  entire  population  (census)  was  surveyed.  However,  52  deans 
responded  the  “deans’  questionnaire”  with  the  return  rate  of  82.5  percent  in  April  and  May  2005. 

Faculty  Members:  The  faculty  members  included  in  this  study  are  the  ones  who  give  “technology 
integration”  courses  to  preservice  teachers.  18  universities,  in  all  regions  of  Nomenclature  of  Units  for 
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Territorial  Statistics  (NUTS)  Level  1 were  selected  by  convenience  sampling  method.  Then,  the  “faculty 
members’  questionnaire”  was  distributed  to  223  faculty  members  from  these  universities  requesting  their 
participation  in  completing  the  questionnaire  in  May  2005.  Ill  faculty  members  responded  the 
questionnaire  with  a return  rate  of  49.8  percent. 

Preservice  Teacher:  The  third  data  sources  used  in  this  study  is  preservice  teachers.  The  4th  year 
preservice  teachers,  who  had  taken  “technology  integration”  courses  before  spring  semester  of  2004-2005, 
were  included  in  this  study.  Approximately,  there  are  33035  4th  year  preservice  teachers  at  teacher 
education  schools  in  Turkey.  The  “preservice  teachers’  questionnaire”  was  distributed  to  2116  preservice 
teachers  from  the  same  18  universities  in  May  2005.  1330  of  preservice  teachers  responded  to  the 
questionnaire  with  the  return  rate  of  62.9  percent. 

Data  Collection  Instruments 

In  this  study,  the  data  were  collected  through  five  different  instruments  which  are  three 
questionnaires  and  two  interview  guides. 

Questionnaire  1 (Ql)  for  Deans  of  Education  Faculties:  The  first  questionnaire  (Ql)  was  developed 
to  collect  data  from  deans  of  teacher  education  schools  and  consisted  of  18-items.  Ql  included  13  multiple 
close-ended  items,  3 five-point  Likert-  type  items,  and  2 open-ended  questions.  Ql  was  developed  by  the 
researchers  based  on  the  related  literature  (Roblyer,  1994;  Vagle,  1995;  Vagle  and  College,  1995; 

Queitzsch,  1997;  South  East  and  Islands  Regional  Technology  in  Education  Consortium,  1998;  SCRTEC, 
1998),  and  issues  investigated  in  this  study. 

Four  experts  (3  deans  and  one  IT  expert)  examined  the  questionnaire  and  based  on  their  suggestions,  the 
questionnaire  was  revised.  Before  the  last  version,  Ql  was  checked  by  a Turkish  Language  expert  for  the  clarity 
of  the  language.  The  items  in  Ql  were  grouped  around  six  major  topics  as  “personal  and  institutional 
information,  ICT  facilities  of  education  faculties,  use  ofICT  in  teaching,  major  barriers  of  integrating  ICT, 
possible  solutions  of  major  barriers  for  integrating  ICT,  and  ICT  skills  of personnel  and  physical  resources 
for  the  education  faculties.  ” In  this  study,  only  the  data  gathered  from  the  two  parts  of  the  questionnaire 
“ major  barriers  of  integrating  IC,  and  possible  solutions  of  major  barriers  for  integrating  ICT”  were  used. 
The  Cronbach  alpha  coefficient  of  the  questionnaire  was  calculated  as  .91  denoting  a satisfactory  reliability. 

Questionnaire  2 (Q2)  for  the  Faculty  Member:  The  second  questionnaire  (Q2)  was  developed  to 
gather  information  from  the  faculty  members  who  give  “technology  integration”  courses  in  the  teacher 
education  programs  in  the  teacher  education  schools  and  consisted  of  24-items.  Q2  included  13  multiple 
close-ended  items,  7 five-point  Likert-  type  items,  and  4 open-ended  questions.  It  was  developed  by  the 
researchers  based  on  research  questions  and  review  of  related  literature  (Topp  et  al.,  1995;  Baron  and 
Goldman,  1995;  and  Imer,  2000). 

Four  experts  examined  the  questionnaire,  and  in  regard  to  the  feedback  gathered  from  the  experts, 
the  instrument  was  revised.  Then,  it  was  checked  by  a Turkish  Language  expert  for  the  clarity  of  the 
language.  After  the  revision,  a pilot  test  was  conducted  with  64  faculty  members.  The  Cronbach  alpha 
coefficient  was  calculated  as  .87  denoting  a satisfactory  reliability.  Subsequently,  a factor  analysis  was 
applied  to  the  scale  whether  the  items  measure  two  factors  basic  ICT  competencies  (factor  1)  and  advanced 
ICT  competencies  (factor  2).  The  items  were  grouped  around  seven  major  topics  {personal  information,  use 
ofICT  in  teaching,  effectiveness  of  the  technology  integration  courses,  perceptions  ofICT  integration, 
major  barriers  of  integrating  ICT,  possible  solutions  of  major  barriers  for  integrating  ICT,  and  ICT  skills 
and  experiences).  In  this  study,  the  data  gather  for  the  two  topics,  “major  problems  of  integrating  ICT"  and 
“solutions  of  major  problems  for  integrating  ICT"  were  used. 

After  gathering  the  data  from  1 1 1 faculty  members,  the  Cronbach  alpha  coefficient  of  the 
questionnaire  was  re-calculated  and  found  as  .97  denoting  a satisfactory  reliability.  Subsequently,  a factor 
analysis  was  applied  to  the  scale  whether  the  items  measure  two  factors.  The  Cronbach  alpha  of  the  Factor  1 is 
.96  and  the  Cronbach  alpha  of  the  Factor  2 is  .92. 

Questionnaire  3 ( Q3 ) for  Preservice  Teachers:  The  third  questionnaire  (Q3)  was  used  to  collect 
data  from  preservice  teachers  and  consisted  of  42 -items.  Q3  included  10  multiple  close-ended  items,  26 
five-point  Likert-  type  items,  and  6 open-ended  questions.  The  questionnaire  was  developed  originally  by 
Tinmaz  (2004),  and  was  adapted  for  this  study.  The  Cronbach  alpha  coefficient  was  calculated  as  .86  and 
split-half  coefficient  was  .91  each  indicating  a pleasing  reliability. 

For  this  study  the  Cronbach  alpha  coefficient  was  re-calculated  as  .91  denoting  a satisfactory 
reliability.  Subsequently,  a factor  analysis  was  applied  to  the  scale  whether  the  items  measure  two  factors  (belief 
of  the  positive  effect  of  technology  in  education,  effect  of  undergraduate  program).  The  Cronbach  alpha  of 
the  Factor  1 is  .93  and  the  Cronbach  alpha  of  the  Factor  2 is  .86. 

Interview:  Semi  structured  interviews  were  conducted  in  this  study  to  collect  in-depth  data  from 
the  faculty  members  and  preservice  teachers.  The  interview  schedules  were  developed  by  the  researchers 
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based  on  research  questions,  topics,  review  of  related  literature  and  format  used  in  previous  studies  by 
Smith  (2002)  and  Zayim  (2004).  The  interview  guides  were  examined  by  four  experts  for  the  clarity  of  the 
questions  and  how  well  they  could  address  the  themes.  After  experts’  review,  pilot  interviews  were 
conducted  with  2 preservice  teachers  and  2 faculty  members  and  necessary  revisions  were  made.  Then,  a 
Turkish  Language  expert  revised  the  interview  schedules  for  the  language  clarification.  The  final  forms  of 
the  each  interview  schedules  included  nine  main  questions.  The  interviews  were  conducted  with  6 faculty 
members  and  6 preservice  teachers  in  May  and  June  2005. 

Data  Analysis 

For  the  quantitative  parts,  descriptive  statistics  were  used  for  data  analysis  to  explore  significant 
problems  and  their  solutions  for  integrating  ICT  into  preservice  teacher  education  programs  in  Turkey  in 
regard  to  the  deans,  faculty  members  and  preservice  teachers.  For  this  purpose,  means,  percentages,  and 
standard  deviations  of  questionnaire  items  were  calculated. 

For  the  qualitative  parts,  the  interviews  were  transcribed  and  after  the  process  of  reading  and  re- 
reading the  transcripts,  researchers  coded  and  analyzed  the  open-ended  data;  then  the  codes  and  themes 
were  analyzed.  Finally,  the  categorized  data  were  compared  based  on  the  main  themes.  The  major  findings 
of  this  study  are  described  in  results  and  conclusion  parts. 

Results 

ICT  Integration  Barriers  Perceived  by  Deans 

Mean  scores  and  standard  deviations  of  the  barriers  perceived  by  deans  were  presented  in  Table  1.  As  it 
is  shown  in  the  Table,  “lack  of  inservice  training  about  ICT’’  (M=4.08)  was  the  most  significant  barrier  faced 
in  integrating  ICT  into  preservice  teacher  education  programs.  The  other  leading  barriers  are  “lack  of 
appropriate  software  and  materials  for  instruction”  (M=3.81),  “lack  of  basic  knowledge-skills”  (M=3.79), 
“lack  of  hardware”  (M=3.72),  “inadequate  repertoire  of  knowledge  and  skills  on  the  integration  of  ICT  in 
instruction”  (M=3.67),  “lack  of  technical  support”  (M=3.60),  which  are  above  the  overall  mean  (M=3.46). 
Following  items  below  the  mean  and  deans  identified  these  statements  as  not  representing  barriers: 
“inappropriate  course  content  and  instructional  programs”  (M=3.38),  “lack  of  time  for  integrating  ICT  in 
classroom-lessons”  (M=2.60),  and  the  lowest  mean  score  was  “inadequate  support  from  upper  positions” 
(2.53). 

Table  1:  The  barriers  faced  in  integrating  ICT  into  teacher  education  schools  according  to  their  deans 
(n=52) 


M 

SI) 

Lack  of  inservice  training  about  ICT 

4.08 

.75 

Lack  of  appropriate  software  and  materials  for  instruction 

3.81 

.93 

Lack  of  basic  knowledge  and  skills  about  ICT 

3.79 

1.06 

Lack  of  hardware  (computer,  printer  etc.) 

3.72 

1.24 

Inadequate  repertoire  of  knowledge  and  skills  on  the 
integration  of  ICT  in  instruction 

3.67 

1.12 

Lack  of  technical  support 

3.60 

1.03 

Inappropriate  course  content  and  instructional  programs 

3.38 

.97 

Lack  of  time  for  integrating  ICT  in  classroom 

2.60 

1.16 

Inadequate  support  from  upper  positions 

2.53 

1.10 

Overall  Mean 

3.46 

Note:  For  this  and  the  following  tables,  1 (Strongly  Disagree),  2 (Disagree),  3 (Neutral),  4 
(Agree),  and  5 (Strongly  Agree) 

ICT  Integration  Barriers  Perceived  by  Faculty  Members 

Means  and  standard  deviations  of  barriers  for  the  faculty  members  were  provided  in  Table  2.  The 
results  indicated  that  faculty  members  perceived  “lack  of  hardware”  (M=4.14)  as  the  most  significant  barrier  in 
integrating  ICT  into  preservice  teacher  education  programs.  The  other  key  barriers  are  “lack  of  appropriate 
software  and  materials  for  instruction”  (M=4.06),  “lack  of  computer  access  for  students’  out-of-class” 
(M=4),  “lack  of  technical  support”  (M=3.99),  “lack  of  inservice  training  about  ICT”  (M=3.95),  “inadequate 
repertoire  of  knowledge  and  skills  on  the  integration  of  ICT  in  instruction”  (M=3.95),  “lack  of  basic 
knowledge-skills  (M=3.94)”,  which  are  above  the  overall  mean  (M=3.79).  The  items  below  the  overall 
mean  include  “the  constraints  related  to  hardware”  (3.77)  and  “lack  of  physical  environment  for  integrating  ICT 
in  classroom”  (M=3.53).  Faculty  members  identified  the  following  statement  as  not  representing  barriers: 
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“inappropriate  course  content  and  instructional  programs”  (M=3 .40)  and  “lack  of  time  for  integrating  ICT  in 
classroom-lessons”  with  the  lowest  mean  score  observed  (M=2.95). 

Table  2:  The  barriers  faced  in  integrating  ICT  into  teacher  education  schools  according  to  their  educators 

(n=l  1 1) 


M 

SD 

Lack  of  hardware  (computer,  printer  etc.) 

4.14 

.99 

Lack  of  appropriate  software  and  materials  for  instruction 

4.06 

.85 

Lack  of  computer  access  for  students’  out-of-class 

4.00 

1.18 

Lack  of  technical  support 

3.99 

1.03 

Lack  of  inservice  training  about  ICT 

3.95 

.95 

Inadequate  repertoire  of  knowledge  and  skills  on  the 
integration  of  ICT  in  instruction 

3.95 

.97 

Lack  of  basic  knowledge  and  skills  about  ICT 

3.94 

1.01 

The  constraints  related  to  hardware 

3.77 

1.11 

Lack  of  physical  environment  for  integrating  ICT  in  classroom 

3.53 

1.11 

Inappropriate  course  content  and  instructional  programs 

3.40 

1.08 

Lack  of  time  for  integrating  ICT  in  classroom 

2.95 

1.17 

Overall  Mean 

3.79 

In  addition  to  the  quantitative  data,  the  findings  from  the  qualitative  data  showed  that  faculty  members 
consider  the  followings  as  ICT  integration  barriers: 
o lack  of  successful  models, 

o inadequate  support  from  upper  position  (administrative  support)  to  the  faculty  member 
who  successfully  integrate  ICT  into  her/his  courses, 
o lack  of  hardware, 

o lack  of  inservice  training  about  ICT, 

o lack  of  technical  support  for  integration  ICT  and  preparation  instructional  materials, 
o inadequate  repertoire  of  knowledge  and  skills  on  the  integration  of  ICT  in  instruction. 

In  addition  to  these  barriers,  the  most  important  problem  for  one  of  the  faculty  member  was  students’ 
attitudes.  The  faculty  member  indicated  that  “We  need  to  change  the  attitude  of  students  in  order  to  benefit  from 
technology  integration  courses ”.  They  have  negative  attitudes  like:  “What  will  I do  it?”  “ Where  will  I use  it?” 
“Why  will  I use  it?”  We  should  change  these  attitudes.  This  is  the  most  important  problem  for  me.” 

ICT  Integration  Barriers  Perceived  by  Preserviee  Teachers 

The  results  of  the  interviews  with  preservice  teachers  and  the  findings  from  the  open-ended  questions  in 
the  preservice  teachers’  questionnaire  complement  each  other.  The  major  problems  indicated  by  preservice 
teachers  are: 

o lack  of  hardware, 

o lack  of  basic  knowledge  and  skills  of  faculty  members  about  ICT, 
o inadequate  knowledge  and  skills  of  faculty  members  for  the  integration  of  ICT  in  their 
classes, 

o need  for  a good  role  model, 

o lack  of  computer  access  in  laboratories  for  students  out  of  the  class, 
o crowded  classrooms, 

o lack  of  computer  and  other  equipments  in  class  for  presentation, 
o negative  attitudes  of  faculty  members  towards  using  ICT  in  their  classes, 
o inadequate  number  of  technology  integration  courses. 

The  barriers  are  grouped  in  two  main  themes  as  “what  they  do  not  have”  and  “what  they  have”  as 
Ertmer  (1999)  suggested.  Under  the  first  group,  the  barriers  indicated  by  preservice  teachers  are  “lack  of 
hardware”,  “lack  of  computer  laboratories  for  free  (out  of  lecture)  time”,  and  “lack  of  computer  and  other 
equipment  in  class  for  presentation”.  In  regard  to  the  first  group  barriers,  one  of  the  preservice  teachers 
said,  “One  month  ago,  I prepared  my  homework  in  CD  format,  but  I could  not  show  it  to  my  teacher  in  class 
because  we  don’t  have  any  computers  in  class.  I want  to  present  my  homework  through  computer  by  using  flash 
animations  and  using  some  pictures.  So  we  have  to  prepare  it  in  traditional  methods.  I think,  at  least  one 
computer  should  be  placed  in  each  classroom.” 
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Preservice  teachers  want  to  have  computer  laboratories  to  use  in  leisure  time,  One  of  the  preservice 
teachers  commented  that  “we  do  not  have  any  computer  labs  for  using  after  the  lesson.  Sometimes,  I have  to  go 
my  home  for  only  checking  my  email.  Every  time,  there  is  lesson  in  computer  laboratories  has  a session  in  class 
or  closed.  We  could  not  use  it  after  the  lessons. 

The  second  group  perceived  barriers  (what  they  have)  were  related  to  “lack  of  basic  knowledge  and 
skills  of  faculty  members  about  ICT”,  “inadequate  knowledge  and  skills  of  faculty  members  for  the 
integration  of  ICT  in  their  classes”,  “need  for  a good  role  model”,  and  “negative  attitudes  of  faculty 
members  towards  using  ICT  in  their  classes”.  One  of  the  preservice  teachers  regarding  faculty  members’ 
attitudes  mentioned  that  “they  don’t  have  any  positive  attitude  towards  computer.  If  they  had,  they  might  be 
able  to  learn  it.  They  could  not  become  a good  model  for  use  of  technology.” 

Possible  Solutions  for  ICT  Integration  Barriers  Perceived  by  Deans 

Means  and  standard  deviations  of  possible  solutions  for  the  deans  were  provided  in  Table  3.  The 
data  explored  that  the  most  strongly  agreed  with  the  possible  solutions  is  “technology  plans  for 
implementing  ICT  in  faculties  and  universities  should  be  prepared”  (M=4.42)  to  integrate  ICT  into 
preservice  teacher  education  programs  as  perceived  by  deans.  The  other  leading  solutions  is  “the  inservice 
training  about  ICT  should  be  improved  in  quantity  and  quality”  (M=4.34),  “more  budget  should  be 
allocated  to  ICT”  (M=4.34),  which  are  above  the  mean  (M=4.25).  Following  items  below  the  mean  are 
“specific  units  and  personnel  should  be  allocated  for  peer  support  and  public  use  of  ICT  tools  and  materials 
to  the  use  of  ICT  in  instruction”  (M=4.24),  ‘the  faculty  members  who  integrate  ICT  in  their  courses  should 
be  supported”  (M=4.24),  “the  course  load  of  the  faculty  members  should  be  decreased”  (M=4.14),  and  the 
lowest  mean  score  was  observed  in  “the  course  content  should  be  redesigned  to  acquire  more  benefit  from 
ICT”  (M=4.08). 

Table  3:  The  possible  solutions  to  current  ICT  integration  barriers  in  preservice  teacher  education 
programs  according  to  deans  (n=52) 


M 

SD 

Technology  plans  for  implementing  ICT  in  faculties  and 
universities  should  be  prepared 

4.42 

.76 

The  inservice  training  about  ICT  should  be  improved  in 
quantity  and  quality 

4.34 

.69 

More  budget  should  be  allocated  to  ICT 

4.34 

.94 

Specific  units  and  personnel  should  be  allocated  for  peer 
support  and  public  use  of  ICT  tools  and  materials  to  the 
use  of  ICT  in  instruction 

4.24 

.85 

The  faculty  members  who  integrate  ICT  in  their  courses 
should  be  supported  (such  as  incentive  payment) 

4.24 

1.02 

The  course  load  of  the  faculty  members  should  be  decreased. 

4.14 

.96 

The  course  content  should  be  redesigned  to  acquire  more 
benefit  from  ICT 

4.08 

.78 

Overall  Mean 

4.25 

Possible  Solutions  for  ICT  Integration  Barriers  Perceived  by  Faculty  Members 

Means  and  standard  deviations  of  possible  solutions  for  the  faculty  members  were  provided  in  Table  4. 
The  data  explored  that  most  strongly  agreed  with  the  possible  solutions  “technology  plans  for  implementing 
ICT  in  faculties  and  universities  should  be  prepared”  (M=4.54)  to  integrate  ICT  into  preservice  teacher 
education  programs  as  perceived  by  faculty  members  as  like  deans.  The  other  top  solutions,  “specific  units 
and  personnel  should  be  allocated  for  peer  support  and  public  use  of  ICT  tools  and  materials  to  the  use  of 
ICT  in  instruction”  (M=4.53),  “more  budget  should  be  allocated  to  ICT”  (M=4.50),  “the  faculty  members 
who  integrate  ICT  in  their  courses  should  be  supported”  (M=4.49),  “the  inservice  training  about  ICT  should 
be  improved  in  quantity  and  quality”  (M=4.44),  which  are  above  the  mean  (M=4.39).  Following  items 
below  the  mean  are  “the  course  content  should  be  redesigned  to  acquire  more  benefit  from  ICT”  (M=4.17), 
and  the  lowest  mean  score  was  observed  in  “the  course  load  of  the  faculty  members  should  be  decreased” 
(M=4.10). 
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Table  4:  The  possible  solutions  to  current  ICT  integration  barriers  in  preservice  teacher  education 
programs  according  to  faculty  members  (n=l  1 1) 


M 

SD 

Technology  plans  for  implementing  ICT  in  faculties  and 
universities  should  be  prepared 

4.54 

.64 

Specific  units  and  personnel  should  be  allocated  for  peer 
support  and  public  use  of  ICT  tools  and  materials  to  the  use 
of  ICT  in  instruction 

4.53 

.62 

More  budget  should  be  allocated  to  ICT 

4.50 

.62 

The  faculty  members  who  integrate  ICT  in  their  courses 
should  be  supported  (such  as  incentive  payment) 

4.49 

.65 

The  inservice  training  about  ICT  should  be  improved  in 
quantity  and  quality 

4.44 

.62 

The  course  content  should  be  redesigned  to  acquire  more 
benefit  from  ICT 

4.17 

.89 

The  course  load  of  the  faculty  members  should  be  decreased 

4.10 

.96 

Overall  Mean 

4.39 

Analyses  of  qualitative  responses  indicated  that  there  are  strongly  agreements  between  the  quantitative 
and  qualitative  results.  In  addition  to  the  quantitative  results,  a faculty  member  suggests  that  the  preservice 
teachers’  motivation  should  been  enhanced.  He  further  offers  new  technology  integration  courses  for  the 
education  faculty.  According  to  him,  two  technology  courses  are  not  providing  for  needs.  Also  one  of  the  faculty 
members  commented: 

Technology  integration  courses  should  be  integrated  to  school  experience  courses.  I think,  this 
model  would  enhance  efficiency  of  integration.  Using  ICT  is  important  but  integration  ICT  in  your 
class  is  more  important.  Also  we  have  to  offer  to  our  students  a new  technology  integration  course 
which  has  to  include  both  ICT  and  field  of  study  (Math,  language,  chemistry)  after  the  second 
method  courses. 

Possible  Solutions  for  ICT  Integration  Barriers  Perceived  by  Preservice  Teachers 

There  are  also  significant  relations  between  the  open-ended  questions  and  interview  results  of 
preservice  teachers  about  the  possible  solutions  to  current  ICT  integration  problems.  The  preservice  teachers 
indicated  solutions  by  stating  the  followings: 

o courses  should  be  supported  by  an  appropriate  web  page 
o faculty  members  should  be  given  inservice  training  for  the  integration  of  ICT 
o course  content  should  be  improved  for  the  today  needs 
o more  hardware  should  be  allocated  to  us 

o our  teacher  should  be  a role  model  to  us  as  using  ICT  in  their  courses 
o more  technology  courses  should  be  offered  to  us 
o every  classroom  should  have  at  least  one  computer 
o every  department  (faculty)  should  have  a free  lab 
o every  technology  integration  courses  should  be  based  on  application 

Interview  results  of  the  preservice  teachers  remarked  below  solutions  to  overcome  current  problems. 
One  of  the  interviewee  suggested  that:  “teachers  should  be  provided  with  inservice  training  for  the  integration 
of  ICT’’.  On  the  other  hand,  another  interviewee  suggested  proficiency  exam  on  ICT  for  the  faculty  members.  If 
a faculty  member  takes  this  exam,  then  he/she  should  be  motivated  (i.e  incentive  payment). 

Implications 

This  study  identified  that  a majority  of  all  stakeholders  believe  “lack  of  in-service  training  about  ICT”, 
“lack  of  appropriate  software  and  materials  for  instruction”,  and  “lack  of  hardware”  are  significant  barriers  for 
integrating  ICT  into  preservice  teacher  education  programs  (see  Table  1 and  2).  There  was  also  agreement 
between  the  stakeholders  on  the  possible  solutions  ranking  “technology  plans  for  implementing  ICT  in  TESs 
and  universities  should  be  prepared”  as  the  highest  among  the  possible  solutions. 

These  barriers  are  consistent  with  findings  Topp  et  al.  (1995),  Baron  and  Goldman  (1994), 
SchoolNetAfrica  (2004),  Ertmer  (1999),  Ertmer,  Addison,  Lane,  Ross,  and  Woods,  (1999),  Ertmer,  Ottenbreit- 
Leftwich,  and  York  (2005),  and  Glazewski,  Ku,  and  Brush  (2001)  have  made.  According  to  Topp  et  al.  (1995) 
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and  Baron  and  Goldman  (1994),  barriers  for  integrating  ICT  into  preservice  teacher  education  are:  1)  limited 
availability  of  equipment;  2)  lack  of  faculty  training;  3)  no  clear  expectation  that  faculty  will  incorporate 
technology  in  academic  activities;  4)  lack  of  funds;  5)  lack  of  time  to  develop  facility  in  using  equipment 
and  software;  6)  doubt  about  the  pedagogical  validity;  7)  lack  of  technical  support;  8)  lack  of  appropriate 
materials;  and  9)  absence  of  clear  programmatic  goals  for  the  teacher  education  program  as  a whole. 

Similar  to  the  above  barriers  from  literature  and  this  study,  the  report  of  SchoolNetAfrica  (2004) 
identified  the  following  barriers:  1)  lack  of  ICT  experience  and  skills  among  teacher  educators;  2)  lack  of 
access  to  technology  in  preservice  training  institutions;  3)  lack  of  access  to  ICT  training  content;  4)  lack  of 
access  to  good  quality  research  (including  content  examples)  from  institutions  that  are  already  integrating 
ICT  into  preservice  training. 

In  a similar  study,  Glazewski,  Ku,  and  Brush  (2001)  also  studied  barriers.  While  findings  of  that  study 
stated  similar  barriers;  they  proposed  “preservice  teachers  did  not  perceive  potential  problems  such  as 
preparation  time  and  implementation  as  major  barriers  to  effective  integration”,  but  “lack  of  or  limited  access  to 
computers  in  schools”,  “not  enough  software  available  in  schools”,  and  “lack  of  knowledge  about  technology”. 
On  the  other  hand,  “faculty  indicated  that  lack  of  preparation  time  and  implementation  time  were  major  reasons 
why  technology  was  not  being  effectively  integrated  in  many  instructional  settings”  (p.  4). 

This  study  also  indicated  a contradict  results  to  the  literature  (OTA,  1995;  US  Department  of  Education, 
2000)  that:  “inappropriate  course  content  and  instructional  programs”,  “lack  of  time  for  integrating  ICT  in 
classroom-lessons”,  and  “inadequate  support  from  upper  positions”  were  below  the  mean,  and  the  majority  of 
deans,  faculty  members,  and  prospective  teachers  identified  these  statements  as  not  representing  barriers. 

In  addition  to  the  aforementioned  barriers,  open-ended  responses  and  the  interview  findings  showed: 
“need  for  a good  role  model  for  prospective  teachers,  lack  of  technology  plans,  lack  of  successful  models  for 
TESs,  crowded  classrooms,  negative  attitudes  and  lack  of  motivation  of  faculty  members,  and  inadequate 
number  of  technology  integration  courses”  are  important  barriers,  which  were  not  identified  in  the  questionnaire. 

In  the  literature  parallel  to  the  study  results,  in  addition  to  ICT  related  courses,  good  role  models  were 
recommended  for  prospective  teachers  to  observe  appropriate  modeling  throughout  their  undergraduate  process 
(SITE,  2002;  Franklin,  1999;  Huang,  1994;  Kariuki,  Franklin,  & Duran,  2001;  Novak  and  Berger,  1991; 
O’Bannon,  Matthew,  & Thomas  1998;  Strudler,  1991,  Yildirim,  1999,  Yildirim,  2000). 

There  was  a general  agreement  between  the  deans’  and  faculty  members’  questionnaire  results  on 
possible  solutions  for  these  barriers.  All  mean  scores  of  two  questionnaires  were  above  4.0.  Thus,  most  of  the 
deans  and  faculty  members  strongly  agreed  on  possible  solutions  to  integrate  ICT  into  preservice  teacher 
education  programs  (see  Table  3 and  4).  There  was  also  agreement  between  the  two  stakeholders  on  the 
possible  solutions,  ranking  “technology  plans  for  implementing  ICT  in  TESs  and  universities  should  be 
prepared”  as  the  most  strongly  agreed  possible  solutions  for  both.  On  the  other  hand,  prospective  teachers’ 
solutions  generally  were  similar  to  what  faculty  members  stated. 

Research  studies  indicate  that  the  implementation  levels  of  technology  into  teaching  and  learning 
remain  low  (Cuban,  2001;  Cuban  et  al.  2001;  Ertmer,  1999;  Olsen,  2001).  Previous  literatures  (Anderson  et  al., 
1998;  Boe,  1989;  Caverly,  Peterson,  & Mandeville,  1997;  Cuban  et  al.  2001;  Vaughan,  2002,  Schoep,  2004),  as 
well  as  the  results  of  this  study,  demonstrate  that  providing  access  to  ICT  is  not  enough;  faculty  members  or 
teachers  need  leadership  and  require  training  in  methods  for  integrating  ICT  into  their  classroom.  As  shown  in 
the  Tables  3 and  4,  the  first  major  solution  is  to  develop  a technology  integration  plan  for  the  TESs. 

According  to  Willis  (2001),  “solutions”  are  local,  not  universal;  however,  the  findings  of  this  study  for 
possible  solutions  are  also  similar  to  the  literature.  Scrimshaw  (2004),  Becker  (1994),  Ertmer  (1999),  Ertmer, 
Addison,  Lane,  Ross,  and  Woods,  (1999),  Ertmer,  Ottenbreit-Leftwich,  and  York  (2005),  Glazewski,  Ku,  and 
Brush  (2001),  Fabry  and  Higgs  (1997),  Hadley  and  Sheingold  (1993),  OTA  (1995),  Topp  et  al.  (1995), 
Japonite  (2001),  Pricewaterhousecoopers  (2001),  Ronnkvist  (1998),  Willis  (1993),  UNESCO  (2002). 

Scrimshaw  (2004)  stated  two  factors  for  possible  solutions  of  major  barriers  of  ICT  usage  and  ICT 
integration  into  education.  One  of  them  is  individual  factors  such  as  the  availability  of  high  quality 
resources,  high  level  of  technical  support,  full  access  to  software  and  hardware  at  all  times,  and  availability 
of  good  quality  training.  The  second  is  school-level  enabling  factors  which  include  a staff  program  of  ICT 
training,  effective  timetabling  of  rooms  and  equipment,  access  to  resources,  on-site  technical  support,  and 
whole-school  policies  on  using  ICT  across  the  curriculum. 

The  following  items  might  also  be  solutions  to  help  overcome  the  significant  barriers  in  the 
literature:  provide  adequate  equipment  and  resources  (Becker,  1994;  Fabry  & Higgs,  1997;  Hadley  & 
Sheingold,  1993;  OTA,  1995;  Topp  et  al.,  1995);  allocate  specific  units  or  personnel  for  peer  support  and  to 
help  reduce  the  teacher  workload  (Becker,  1994;  Japonite,  2001;  OTA,  1995;  Pricewaterhousecoopers 
2001;  Ronnkvist,  1998;  Sandholtz,  2004);  staff  development  (Odabasi,  2000;  OTA,  1995;  Willis,  1993); 
preparation  of  technology  plans  for  implementing  ICT  in  TESs  and  universities  (UNESCO,  2002). 
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Abstract 

This  paper  presents  the  findings  from  the  most  recent  data  set  of  an  ongoing  naturalistic  study  designed  to  study 
communication,  interaction,  and  learning  strategies  in  an  online  learning  environment.  Analysis  of  past  and  current 
data  has  suggested  that  the  use  of  ancillary  communication  techniques  provide  increased  instructional  effectiveness 
and  learner  satisfaction  in  online  classes.  Ancillary  communication  as  an  intentional  instructional  strategy  will  be 
introduced.  A model  is  proposed  and  strategies  and  recommendations  for  its  use  will  be  presented. 


Introduction 

Internet-based  classes  appear  to  dominate  the  landscape  of  distance  education  today.  Online  classes  are 
common  place  in  most  colleges  and  universities  and  are  becoming  more  common  in  high  schools  as  well.  While 
pioneering  courses  were  offered  through  techniques  using  email,  Internet  Relay  Chat  (IRC)  or  bulletin  boards, 
today’s  courses  are  almost  all  managed  by  portal  systems  such  as  WebCT™,  or  Courselnfo™.  These  systems  share 
many  communication  features  such  as  bulletin  boards,  email,  chat,  whiteboards  and  assignment  drop  boxes.  As 
technology  advances  and  bandwidth  improves,  we  see  changes  in  these  environments.  Today’s  systems  are 
beginning  to  offer  audio  and  video  communication  tools  as  well  as  traditional  text  based  communication.  These  new 
tools  provide  us  with  a unique  opportunity.  No  longer  is  an  online  environment  constrained  to  text  for  synchronous 
meetings  and  packaged  media  for  asynchronous  meetings.  The  environment  now  affords  unique  opportunities  for 
communication  patterns  that  are  usually  reserved  for  face-to-face  classes,  mainly  the  ability  to  talk  and  listen. 

Moreover,  with  the  addition  of  audio  and  video  we  are  able  to  add  new  channels  of  information  to  the 
online  environment.  Historically  there  has  been  much  research  done  on  multiple-channel  communication  and  cue 
summation.  Moore,  Burton,  and  Myers  (2004)  provide  an  exhaustive  review  of  the  literature  on  both.  At  the  risk  of 
oversimplifying  this  review,  we  can  say  that  on  the  positive  side  multiple-channels  of  information  may  provide 
greater  enrichment  in  learning.  On  the  negative  side  it  may  lead  to  cognitive  overload  for  learners. 

In  a naturalistic  online  environment  we  have  studied  the  use  of  these  communication  tools  and  found  that 
while  they  may  be  used  to  provide  redundancy  of  information  and  enrichment  of  material,  (in  the  manner  of 
traditional  multiple-channels  or  cue  summation)  they  can  be  used  another  way  as  well.  They  may  be  used  as  support 
communication,  such  as  chat  supporting  audio,  or  audio  supporting  whiteboards,  or  audio  supporting  chats.  When 
used  in  a purposeful  manner  these  multiple  channels  may  create  increased  learner  focus  and  more  efficient 
communication.  In  traditional  positivist  learning  environments  the  primary  instructional  communication  occurs 
between  the  teacher  and  the  students.  In  constructivist  learning  environments  the  primary  instructional 
communication  may  occur  between  the  instructor  and  students  or  between  students  and  students.  In  either 
environment  the  instructional  emphasis  of  the  communication  is  almost  wholly  on  one  or  the  other.  We  argue  that 
potential  learning  benefits  may  accrue  if  we  place  more  emphasis  on  the  communications  that  are  occurring  in  a 
learning  environment  simultaneously  with,  but  outside  the  focus  of  the  primary  communications.  We  call  the 
exchange  of  information  in  support  of  learning  that  occurs  synchronously  with,  but  is  physically  and  semantically 
separate  from,  a primary  communication  mode  ancillary ; communication. 

Theoretical  Perspectives 

The  design  of  a course  using  the  ancillary  approach  is  based  on  the  notion  of  constructivist  learning 
(Jonassen,  1999),  situated  cognition  (Brown,  Collins  & Duguid,  1989)  and  anchored  instruction  (Bransford  et  al, 
1990;  Cognition  and  Technology  Group,  1990).  In  the  spring  of  2006  we  developed  and  offered  a graduate  level 
class  about  online  learning  to  take  advantage  of  new  and  emerging  theories  and  practice  in  the  field.  The  class  was 
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set  up  to  be  experiential  and  to  create  a learning  environment  that  is  driven  by  the  learner  (Papert,  1990;  Wilson  & 
Ryder,  1996;  Greening,  1998).  Analysis  of  prior  offerings  of  the  class  indicated  that  students  were  unprepared  for 
the  responsibility  of  this  type  of  experience  initially  and  typically  floundered  during  the  early  portion  of  the 
semester.  To  counter  this  we  employ  a pedagogical  shift  approach,  from  a more  positivist  perspective  and  instructor 
delivered  content  early  in  the  semester,  to  a more  constructivist  perspective  with  student  generated  content  and 
instructor  guidance  later  in  the  semester.  This  shift  allows  the  students  to  quickly  gain  confidence  in  the  online 
environment  and  master  the  fundamental  knowledge  and  skills  they  will  need  to  succeed  later  on,  while  still 
affording  them  the  benefits  usually  conferred  by  a student-centered  learning  environment  (i.e.  greater  motivation 
and  enhanced  transfer)  (Land,  S.  & Hannafin,  M.,  2001).  This  is  particularly  important  since  motivation  has  been 
found  to  be  a key  component  for  successful  online  learning  (Kawachi,  2003). 

This  work  draws  heavily  upon  a taxonomy  of  web-based  instruction  developed  by  Harmon  and  Jones 
(1999),  research  focusing  on  analyzing  learner  interaction  in  online  classes  (Harmon  & Jones,  2001)  and  discourse 
patterns  in  online  discussions  (Mcklin,  Harmon,  Jones  & Evans,  2002).  Additionally  it  is  related  to  the  foundational 
research  in  communications  theory  (Schramm,  1961;  Shannon  & Weaver,  1964)  and  multiple-channel 
communication  and  cue  summation  (Moore,  Burton,  & Myers,  2004). 

Ancillary  Communication 

Historically  the  class  has  been  offered  through  a portal  system  such  as  WebCT™  or  Courselnfo™. 
Consequently  our  communication  tools  were  bulletin  boards,  email,  and  drop  boxes  for  asynchronous  meetings  and 
IRC  for  synchronous  meetings.  During  2004  we  added  a course  system  called  Elluminate™  which  provided  two 
way  audio  communications  between  students  and  students  and  faculty  and  students,  and  faculty  and  faculty.  We  also 
added  web-based  video  conferencing  tools,  which  were  not  originally  a part  of  Elluminate™,  for  faculty  interaction 
during  class  and  for  planning  for  the  class.  The  system  provided  space  for  posting  packaged  PowerPoint™ 
presentations  and  also  provided  the  ability  to  break  students  into  small  groups  in  the  synchronous  meetings  and 
provide  them  with  audio  tools  and  whiteboard  space  for  group  work  during  class  meetings.  As  instructors  we  were 
able  to  see  all  communications  in  the  class,  even  those  that  were  “private”  to  the  students.  Students  were  given 
problems  during  class,  such  as  doing  the  front  end  analysis  for  a web-site  for  a client,  in  a small  group  and  then 
brought  back  together  with  the  larger  class  to  present  their  findings.  Student  interactions  and  discourse  using  these 
tools  and  strategies  were  analyzed  both  at  the  small  group  level  and  the  whole  class  level.  Initially  we  found  that 
these  tools  were  not  technically  difficult  for  people  to  use,  but  that  students  were  not  clear  on  why  they  should  use 
them  and  how  they  could  be  best  used.  Through  the  use  of  recursive  data  analysis  we  began  to  develop  strategies  to 
use  these  new  tools.  What  we  found  was  that  they  were  best  used  when  one  communication  tool  or  mode  was  used 
to  support  the  other. 

During  an  interaction,  a dominant  mode  might  arise.  For  example,  a small  group’s  white  board  (a  visual 
mode)  might  be  perceived  to  be  the  dominant  mode.  To  support  this,  students  may  explain  it  using  an  ancillary 
mode,  such  as  the  audio  functions,  (a  verbal  - audio  mode)  while  taking  questions  from  faculty  and  other  students 
through  another  ancillary  mode,  the  chat  window  (a  verbal  - text  mode).  The  dominant  and  ancillary  communication 
modes  might  change  without  notice,  but  participants  in  the  system  were  able  to  recognize  and  adapt  to  this  almost 
immediately.  They  were  then  able  to  use  ancillary  communication  modes  to  support  the  dominant  mode. 
Consequently,  audio  was  used  to  support  whiteboards  and  vice  versa  depending  on  which  mode  was  perceived  to  be 
dominant.  At  no  point  was  one  mode  stated  to  be  dominant.  The  class  appears  to  be  able  to  develop  a shared 
understanding  of  which  mode  is  dominant  and  which  mode  is  ancillary. 

But  what  we  found  most  interesting  was  that  people  were  not  confused  by  the  tools  or  by  what  sounds  like 
a barrage  of  information.  They  were  actually  engaged  by  the  communication  and  able  to  make  sense  out  of  multiple 
streams  of  information  coming  at  them  at  one  time.  Evidence  of  this  is  found  both  in  their  test  scores  and  in  the 
quality  and  quantity  of  their  interactions.  Consequently,  data  from  the  study  suggests  initially  that  this  is  more  than 
enrichment  or  cue  summation,  especially  since  the  ancillary  communications  often  varied  widely  from  the  content 
being  discussed  in  the  primary  channel.  These  communication  tools  are  seen  by  participants  as  being  interconnected 
in  one  sense,  but  separate  in  another.  Therefore  their  mutual  support  of  each  other  makes  them  unique  and  allows  us 
to  focus  on  their  use  as  an  intentional  instructional  strategy. 

In  addition,  we  find  that  students  in  the  course  interact  in  a manner  consistent  with  the  types  of  interaction 
noted  by  Miltiadou  & Savenye  (2003).  Learner-content  interactions  occur  when  students  in  the  course  work  directly 
with  course  materials.  These  types  of  interactions  are  perhaps  the  most  common  when  viewed  from  the  perspective 
of  time  on  task,  both  in  traditional  and  online  environments.  Reading  a text  book  or  website,  researching  a paper,  or 
working  through  a tutorial  would  all  be  examples  of  learner  content  interaction.  This  type  of  interaction  can  occur 
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concurrently  with  ancillary  communications  and  frequently  does  so  in  our  classes.  However,  we  have  not  yet 
formally  considered  it  as  a part  of  our  emerging  model. 

Learner-instructor  interactions  occur  when  the  student  works  directly  with  the  instructor.  These  types  of 
interactions  are  also  very  common  in  both  traditional  and  online  classes,  but  may  frequently  be  characterized  by  a 
greater  flow  of  information  in  pedagogical  approaches  that  view  the  instructor  as  content  provider.  Interestingly, 
direct  learner-instructor  interactions  may  be  less  common  in  the  type  of  eLearning  environments  that  have  come  to 
dominate  education  with  the  advent  of  course  management  systems  (CMS).  While  in  traditional  CMS  learner- 
instructor  interaction  typically  occurs  both  on  bulletin  boards  and  in  chat  rooms,  there  is  also  a greater  prevalence  of 
stand  alone  instruction  in  which  there  is  little  or  no  direct  learner-instructor  interaction. 

In  the  virtual  classroom  environment  which  prompted  the  development  of  this  model  however,  we  found 
there  to  be  copious  and  intense  learner-instructor  interaction.  Indeed,  we  find  the  level  of  learner-instructor 
interaction  to  frequently  exceed  that  which  is  found  in  a face-to-face  environment.  We  have  noted  four  sub- 
categories of  learner-instructor  interaction  in  this  environment.  In  the  first  sub-category  the  instructor  is 
communicating  with  the  entire  class  at  once  through  audio  and  visual  channels.  This  sub-category  interaction  may 
be  most  easily  thought  of  as  an  instructor  presenting  a lecture  via  PowerPoint.  Note  though,  that  while  this  is  the 
easiest  way  to  characterize  the  interaction,  it  is  also  a bit  misleading  since  pure  lecture  is  a technique  we  try  to 
employ  rarely,  preferring  instead  a more  interactive  form  of  discourse.  The  second  sub-category  of  learner- 
instructor  interaction  occurs  when  the  instructor  is  communicating  with  the  class  as  a whole  through  the  chat 
window.  This  type  of  interaction  is  a central  component  of  the  ancillary  communication  model  we  are  proposing. 
The  third  sub-category  of  learner-instructor  interaction  occurs  with  the  instructor  communicating  with  an  individual 
student  or  a small  group  of  students  privately  in  the  chat  window.  By  privately  we  mean  that  the  communication  is 
not  seen  by  the  class  as  a whole,  only  by  the  intended  recipients.  The  fourth  type  of  sub-category  is  what  we  refer  to 
as  a pseudo-private  communication,  in  which  the  instructor  communicates  with  the  class  as  a whole  via  the  chat 
window,  but  does  so  as  a private  communication.  This  means  that  each  recipient  may  think  that  the  communication 
was  intended  only  for  him  or  her  even  though  in  fact  the  entire  class  got  the  same  communication. 

The  third  type  of  interaction  noted  by  Miltiadou  & Savenye  (2003)  is  learner-learner  interaction.  This 
occurs  when  learners  communicate  directly  with  each  other.  This  type  of  interaction  may  occur  in  traditional 
classrooms,  particularly  those  that  are  set  up  along  constructivist  lines,  or  may  occur  perhaps  surreptitiously  in  a 
classroom  set  up  with  a direct  instruction  format.  In  a traditional  online  classroom,  this  type  of  interaction  may 
occur  more  often  than  in  a traditional  face  to  face  classroom,  particularly  with  student  interaction  that  might  occur 
on  a discussion  board,  or  in  a chat  room.  We  observe  that  like  learner-instructor  interaction,  there  are  sub-categories 
for  this  type  of  interaction  in  the  virtual  classroom.  These  include,  learners  communicating  with  the  class  as  a whole 
via  audio  and  visual  channels;  learners  communicating  with  the  class  as  a whole  via  the  chat  window;  and  learners 
communicating  with  other  individual  or  small  groups  of  learner's  via  private  chat.  Note  that  in  both  the  learner- 
instructor,  and  the  learner-learner  sub-categories  we  suppose  that  there  could  be  an  additional  category  of  accidental 
communications,  in  which  either  the  learner  or  the  instructor  sends  a private  message  that  was  supposed  to  the 
public,  or  more  commonly,  a public  message  that  was  intended  to  be  private.  For  the  purposes  of  this  paper  we  will 
not  consider  these  unintentional  communications. 

The  fourth  type  of  interaction  noted  by  Miltiadou  & Savenye  (2003)  is  learner-interface  interaction. 

Unlike  the  other  three  types  of  interaction  which  may  have  learning  as  their  primary  purpose,  learner-interface 
interaction  has  as  its  primary  purpose  the  enabling  of  the  other  types  of  interactions.  While  it  may  be  possible  to 
design  online  learning  environments  in  which  the  interface  itself  is  intended  as  a mechanism  to  enhance  instruction 
and  learning,  we  know  of  no  course  management  system  or  virtual  classroom  environment  in  which  this  is  the  case. 

A Model  for  Ancillary  Communication 

We  observed  ancillary  communications  occurring  naturally  in  the  virtual  classroom  environment.  We  see 
this  type  of  communication  as  an  affordance  of  virtual  environments  that  may  not  be  so  readily  available  in 
traditional  face-to-face  environments.  Ancillary  communication  is  the  exchange  of  information  in  support  of 
learning  that  occurs  synchronously  with,  but  is  physically  and  semantically  separate  from  a primary  communication 
mode.  We  include  an  “exchange  of  information”  in  the  definition  to  indicate  that  the  communication  should,  at  least 
to  some  extent,  be  two-way.  We  include  “in  support  of  learning”  primarily  to  provide  context  for  our  work  in  this 
area.  We  do  note  that  ancillary  communication  may  not  necessarily  be  in  support  of  learning,  but  that  is  the  only 
form  with  which  we  are  concerned.  We  note  that  ancillary  communication  “is  physically  separate”  from  a primary 
communication  mode  in  order  to  differentiate  it  from  multiple  strands  of  communication  occurring  simultaneously 
on  the  same  channel  (i.e.  several  people  talking  to  you  at  once).  We  include  “semantically  separate”  to  differentiate 
ancillary  communication  from  multiple  channel  communication.  Specifically,  ancillary  communication  is  not 
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intended  to  provide  the  same  exact  information  that  is  coming  through  the  primary  communication  mode,  not  even 
necessarily  to  provide  redundancy  in  the  primary  communication.  Instead  it  is  intended  to  augment  the  primary 
communication  in  ways  that  enhance  learning. 

Ancillary  communication  may  be  either  public,  meaning  that  it  is  made  to  and  intended  for  the  entire  class. 
Examples  of  public  communication  might  be  a description  of  a phenomenon  such  as  occurs  in  a traditional  lecture. 
Ancillary  communication  may  also  be  private,  meaning  that  it  is  made  to  one  specific  person  or  to  a group  of 
specific  people.  Teachers  may  converse  privately  with  each  other  to  plan  an  activity  or  to  respond  to  something  they 
see  in  the  learning  environment.  A teacher  may  answer  a question  posed  by  a student  privately  because  the  student 
needs  remediation  or  because  the  issue  is  not  intended  for  the  entire  class.  Students  may  communicate  privately  with 
each  other  to  clarify  a point  in  the  class  or  to  discuss  an  issue  that  is  not  germane  to  the  class  discussion.  Ancillary 
communication  might  also  be  pseudo-private,  or  sent  out  to  the  group  at  large  as  if  it  were  meant  to  be  a private 
message  even  though  everyone  in  the  class  might  get  the  message.  This  strategy  can  be  useful  for  a teacher  if  there 
is  a question  that  has  been  asked  by  one  or  more  students  but  the  answer  could  be  germane  to  the  entire  class.  It  may 
be  sent  as  if  it  were  private  so  as  not  to  interrupt  a current  discussion  or  presentation  to  the  entire  class.  Any  member 
of  the  learning  community,  either  a teacher  or  a student,  has  the  option  of  using  public,  private,  or  pseudo-private 
messages  at  any  time.  A member  of  the  learning  environment  will  make  a conscious  decision  about  whether  or  not 
message  is  public,  private  or  pseudo-private  depending  on  the  needs  of  the  communication. 

With  a public,  private  or  pseudo-private  message  a member  of  the  learning  environment  may  decide  to 
make  a message  positive  or  negative.  While  students  may  make  positive  and  negative  comments  about  the  course, 
teachers  can  use  positive  and  negative  pieces  of  ancillary  communication  for  a particular  effect.  For  example 
teachers  may  agree  with  each  other  privately  but  disagree  with  each  other  publicly  in  order  to  create  a discussion 
point  or  to  intentionally  create  cognitive  dissonance.  Teachers  may  send  positive  comments  to  students  to  encourage 
them  or  to  let  them  other  wise  know  that  they  are  on  the  right  track.  Teachers  may  send  negative  communication  to  a 
student  to  correct  a mistake  in  an  area  such  as  content,  application,  or  logic.  Application  of  this  model  for  ancillary 
communication  may  be  made  more  specific  through  the  use  of  a variety  of  methods  of  ancillary  communication. 

Methods  of  Ancillary  Communication 

Using  ancillary  communication  in  an  online  environment  can  be  made  more  concrete  through  the 
application  of  one  or  more  method  as  listed  in  Table  1 and  discussed  in  the  narrative  below. 

Table  1 

Methods  of  Ancillary  Communication 


Agree-disagree 

Elaborate 

Diverge 

Scaffolding 

Reiterate 

Emphasis 

Show  relevance  of 

Social  Engineering 

While  there  are  many  potential  methods  of  employing  ancillary  communication  and  online  learning  we  will 
briefly  review  eight  of  these  we  have  attempted  to  use  in  our  own  work.  We  note  at  the  outset  that  our 
circumstances  may  be  a bit  uncommon  in  that  we  are  fortunate  to  have  two  instructors  present  in  the  classroom  at 
the  same  time.  The  methods  of  ancillary  communication  we  employ  typically  take  the  form  of  one  instructor  being 
responsible  for  the  primary  communication  which  is  occurring  in  audio  and  visual  channels,  and  the  other  instructor 
responsible  for  the  ancillary  communication  which  occurs  in  the  chat  window. 

Agree-disagree 

In  the  first  method  the  ancillary  instructor  either  agrees  or  disagrees  with  what  the  primary  instructor  is 
saying  via  the  audio  channel.  The  intention  is  to  either  affirm  the  content  and  thus  add  weight  to  it,  or  to  stimulate 
discussion  and  more  critical  thought  by  disagreeing  with  the  primary  instructor.  We  found  that  it  is  often  startling 
for  the  students  to  see  two  instructors  disagree  with  each  other  in  front  of  the  class.  The  disagreement  forces 
students  to  think  more  deeply  about  the  content  and  to  come  to  some  resolution  as  to  their  own  perspective.  We 
began  using  this  method  after  we  noticed  that  students  frequently  view  a communication  from  the  instructor  as  the 
final  word  on  a subject.  We  found  that,  particularly  on  discussion  boards,  one  of  the  quickest  ways  to  end  a 
discussion  is  to  have  one  of  the  instructor's  offer  an  opinion  on  the  matter.  We  also  found  in  practice  we  typically 
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disagree  little  with  each  other.  Therefore  in  order  to  use  this  technique  it  is  necessary  for  one  of  us  to  play  devil’s 
advocate,  but  do  so  in  a manner  that  is  convincing  to  the  students. 

Elaborate 

In  this  technique  the  ancillary  instructor  elaborates  on  the  primary  instructor's  point.  Consistent  with 
Reigeluth’s  (Reigeluth,  1983)  elaboration  theory  of  instruction  (Reigeluth  & Stein,  1983),  we  found  it  helpful  on 
both  a macro  and  microlevel  to  use  ancillary  communication  as  an  elaboration  tool.  That  is  to  say,  the  ancillary 
instructor  offers  comments  that  place  the  primary  instructor's  point  in  a broad  context,  and  at  times  zooms  in  to 
focus  on  a particular  aspect  of  the  primary  instructor's  point. 

Diverge 

This  technique,  drawn  from  work  in  creativity  theory,  (Torrance,  1985)  consists  of  having  the  ancillary 
instructor  offer  comments  that  may  initially  not  seem  related  to  the  primary  instructor's  comments.  We  find  this 
technique  to  be  most  helpful  when  we  are  trying  to  get  students  to  think  about  unfamiliar  concepts  in  a new  way.  It 
is  a difficult  technique  to  employ  skillfully  since  with  too  much  divergence  the  students  may  become  confused  and 
lose  track  of  the  main  content  object.  However,  we  have  achieved  good  results  by  having  the  ancillary  instructor 
begin  the  divergence  and,  after  sometime  for  reflection,  the  primary  instructor  show  the  relevance  of  the  divergence 
to  the  main  content  object. 

Scaffolding 

In  this  technique  the  ancillary  instructor  assists  the  class  in  comprehending  conceptually  difficult  topics  by 
providing  both  subordinate  skills  necessary  to  comprehend  the  topic,  and  examples  and  non  examples  that  may 
assist  in  understanding.  The  ancillary  instructor  may  also  answer  individual  student’s  questions  either  publicly  or 
privately  depending  on  how  relevant  they  seem  to  the  class  as  a whole. 


Reiterate 

This  brief  technique  occurs  when  the  ancillary  instructor  reiterates  a point  the  primary  instructor  is  making. 
We  use  it  mainly  for  emphasis  and  to  increase  focus  on  particularly  important  points.  In  reiteration,  we  typically  try 
to  use  the  exact  language  that  was  used  by  the  primary  instructor. 

Emphasis 

Another  technique  we  use  frequently  is  to  either  emphasize  or  occasionally  de-emphasize  points  the 
primary  instructor  has  made.  Unlike  reiteration  emphasis  typically  does  not  use  the  same  language  as  the  primary 
instructor.  Instead  it  may  take  the  form  of  approbation  such  as  "Good  point;  that’s  very  important."  De-emphasis 
occurs  less  frequently  but  is  used  when  students  may  have  a tendency  to  place  too  much  importance  on  a particular 
point. 

Relevance 

We  frequently  find  it  helpful  for  the  ancillary  instructor  to  make  explicit  the  relevance  of  the  content.  Even 
more  helpful  is  for  the  ancillary  instructor  to  guide  the  class  in  providing  their  own  examples  of  the  relevance  of  the 
content  during  the  primary  instruction.  We  have  considered,  but  not  yet  attempted,  having  the  ancillary  instructor 
work  through  all  phases  of  Keller's  ARCS  (1987)  model  at  appropriate  points  in  a lesson. 

Social  engineering 

In  this  category  we  include  any  attempt  by  the  ancillary  instructor  to  help  set  the  tone  or  mood  of  the  class. 
This  might  include  adding  humor  to  the  content,  building  confidence  in  the  learners,  or  perhaps  creating  tension  in 
the  class.  In  other  words,  social  engineering  involves  modifying  the  emotional  state  of  the  class  to  best  fit  the 
instructional  purposes  of  the  lesson. 


Conclusion 

We  can  expect  new  communications  technologies  and  increases  in  bandwidth  to  be  upon  us  quickly.  We 
need  to  be  experimenting  with  this  technology  and  applying  what  we  already  know  about  teaching  and  learning  to 
the  use  of  the  technology.  We  propose  that  purposefully  including  ancillary  communications  as  an  instructional 
strategy  in  online  courses  may  enhance  retention  and  transfer  of  content,  and  increase  student  motivation  and 
instructor  feedback. 
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Ancillary  communication  allows  students  to  explore  aspects  of  the  course  content  not  covered  in  the 
primary  instructional  communication  mode,  and  thus  potentially  enhances  their  motivation  in  understanding.  In 
addition,  it  provides  the  instructors  with  "real-time"  feedback  on  course  activities,  allowing  us  to  respond  more 
effectively  to  changing  conditions  for  learning.  To  be  sure,  it  takes  some  time  to  become  comfortable  working  in  an 
ancillary  communications  environment,  and  the  rapid  pace  and  semantically  intense  nature  of  ancillary 
communication  may  not  be  for  everyone.  Yet  our  initial  experience  indicates  that  ancillary  communication  may  be 
an  important  affordance  of  online  instruction;  one  that  could  do  much  to  take  advantage  of  the  inherent  power  of 
modern  computer  mediated  communications.  As  we  grow  more  accustomed  to  dealing  with  ever  increasing 
information  flows,  and  as  a new  generation  of  learners  come  to  our  classrooms,  ancillary  communication  may  have 
an  important  role  to  play  in  the  future  of  online  learning. 
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Introduction 

Gaining  tenure  status  is  one  of  the  hallmarks  of  achievement  for  a faculty  member.  It  implies  recognition, 
acknowledgement,  and  affirmation  from  one’s  professional  colleagues.  However,  the  process  of  achieving  a tenure 
rank  is  a rite-of-passage  fraught  with  pains,  challenges,  and  frustrations.  As  we  reflected  upon  the  process  of 
developing  our  tenure  portfolios  three  essential  questions  evolved:  1)  How  should  beginning  faculty  members 
prepare  for  tenure  in  such  a way  that  minimizes  the  challenges?  2)  What  constitutes  an  effective  presentation  of 
one’s  work  that  will  enhance  the  evaluation  of  that  work?  and  3)  Can  the  process  of  creating  a professional  portfolio 
be  a reflective  opportunity  for  professional  growth?  We  suggest  it  is  possible  that  a quality  electronic  professional 
portfolio  may  enable  us  to  achieve  these  goals. 

Yet,  the  dilemma  is  where  to  begin  and  how  to  proceed  into  this  uncharted  “tenure-seeking”  journey.  Each  year, 
university  faculty  face  the  arduous  task  of  documenting  their  accomplishments  for  annual  review,  tenure,  and 
promotion.  While  most  higher  education  institutions  provide  guidelines  for  these  evaluations,  faculty  go  through  the 
rite-of-passage  virtually  on  their  own  with  minimal  support.  At  best  the  “tenure-seeking”  journey  is  a twisting  road 
with  potholes  of  uncertainty  and  frustration,  where  the  travelers  wish  they  had  some  type  of  map. 

With  our  tenure  journey  still  fresh  in  our  minds,  we  decided  to  write  an  article  to  assist  those  who  are  seeking 
tenure.  While  no  one  person  or  one  article  can  guarantee  tenure,  it  is  our  hope  that  these  guidelines  will  make 
pathways  clearer  and  the  journey  less  stressful  and  more  professionally  affirming. 

Background 

In  education,  portfolios  are  used  at  several  levels  to  evaluate  learning,  competencies,  and  accomplishments.  In 
public  schools,  the  use  of  the  portfolio  is  seen  as  a valid  and  comprehensive  assessment  of  student’s  authentic 
performance.  In  some  teacher  education  programs,  portfolios  are  used  to  evaluate  the  budding  teacher’s  attainment 
of  competencies  (Lynch  & Purnawarman,  2004).  Additionally,  many  states  use  professional  portfolios  to  evaluate 
in-service  teachers.  At  the  university  level,  portfolios  have  long  been  used  to  evaluate  faculty  for  tenure  and 
promotion.  Traditionally,  tenure  portfolios  have  been  notebooks  with  collections  of  artifacts  with  some  level  of 
organizing  to  assist  the  audience.  However,  the  production  of  paper  portfolios  becomes  so  daunting  that  the  author 
loses  sight  of  the  portfolio’s  higher  purposes  (Campbell,  Cignetti,  Melenyer,  Nettles,  & Wyman,  2004;  Martin- 
Kniep,  1999). 

With  technology,  the  process  of  documenting  professional  achievements  has  become  more  flexible,  even  though 
still  labor-intensive.  Rather  than  turning  several  pages  to  see  supportive  evidence  of  a competency  area,  hyperlinks 
take  us  directly  to  the  artifacts  and  help  to  establish  an  immediate  link  and  reinforce  the  conceptual  connection 
between  the  two.  As  the  technology  evolves,  portfolios  have  changed  and  developed  into  more  than  just  a collection 
of  artifacts,  it  has  become  a learning  endeavor  (Constantino  & De  Lorenzo,  2002;  Easley  & Mitchell,  2003). 

Teaching  professionals,  whether  at  the  novice,  in-service,  or  university  level,  struggle  to  capture  the  essence  of 
their  teaching.  Teaching  is  both  an  art  and  a science,  a complex  dynamic  endeavor  that  is  difficult  to  describe.  In 
discussing  teacher  portfolios,  Evans  (1995)  concludes  that  “a  professional  portfolio  is  an  evolving  collection  of 
carefully  selected  and  composed  professional  thoughts,  goals  and  experiences  that  are  threaded  with  reflection  and 
self-assessment.  It  represents  who  you  are,  what  you  do,  and  why  you  do  it,  where  you  have  been,  where  you  are, 
and  where  you  want  to  go,  and  how  you  plan  on  getting  there.”  (p.  11).  As  portfolios  become  accepted  as  valid 
assessment  tools  in  the  schools,  more  teacher  educators  are  advocating  that  future  and  current  teachers  develop 
professional  portfolios  as  a way  to  consolidate  their  learning  experiences. (Harris  & Aworuwa,  2005a;2005b)  We 
advocate  that  teacher  educators  develop  and  maintain  professional  portfolios  to  enhance  a deeper  understanding  of 
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their  own  profession.  While  the  use  of  portfolios  to  evaluate  higher  education  faculty  is  not  new,  the  use  of 
electronic  portfolios  offers  new  and  exciting  dimensions  to  the  faculty  evaluation  process. 


Creating  a Tenure  Portfolio 

A tenure  portfolio  can  be  likened  to  what  Painter  and  Wetzel  (2005)  called  an  assessment  portfolio  which 
presents  educational  organizations  with  information  about  a teacher’s  effectiveness.  Higher  educational  institutions, 
in  particular,  expect  faculty  members  to  demonstrate  competence  in  three  major  categories  including  teaching, 
scholarship,  and  service  to  the  community.  Faculty  that  are  deemed  competent  in  those  desired  areas  are  likely  to 
achieve  tenure  status.  A faculty  member  seeking  tenure  should  use  their  portfolio  as  an  opportunity  to  demonstrate 
competencies  in  all  major  areas  in  which  they  would  be  judged. 

Whether  paper  or  electronic,  the  challenge  is  how  to  represent  the  depth  and  breadth  of  your  professional  work  to 
your  audience.  Each  of  the  three  essential  components,  teaching,  scholarship,  and  service,  assessed  during  a tenure 
review,  have  inherent  complexities  that  are  difficult  to  completely  represent  on  paper  or  with  lists.  With  technology, 
the  process  of  documenting  competence  in  professional  activities,  even  though  still  labor-intensive,  has  become 
more  flexible.  Creating  digital  or  electronic  portfolios  remove  the  constraints  of  paper-based  collections,  allowing 
the  author  to  easily  customize  their  portfolio,  and  include  digital  videos,  audios,  and  graphics.  Portfolios  can  also  be 
stored  on  compact  discs,  which  removes  the  burden  of  carrying  about  heavy  paper  folios  (Barret,  2001).  In  addition, 
“electronic  portfolios  often  have  a different  look  than  print  portfolios  because  the  [audience  can]  navigate  them  with 
hyperlinks”  (Painter  and  Wetzel,  2005)  and  follow  the  flow  of  information  in  any  order.  It  is  the  flow  of  an 
electronic  portfolio  that  allows  the  creator  to  lead  the  reviewers  to  a window  revealing  the  thoughts  behind  the 
works  presented. 

Our  purpose  is  to  provide  a guideline  designed  to  give  you  both  practical  and  conceptual  assistance  in 
constructing  a professional  tenure  portfolio  that  enables  you  to  highlight  your  accomplishments;  reflect  on  the 
congruency  of  your  philosophies,  values,  and  practice;  and  shape  your  future  directions.  From  our  own  tenure 
seeking  experience,  we  have  gleaned  six  steps  that  can  be  used  to  guide  tenure  preparation.  They  include:  1) 

Prepare  for  Success;  2)  Use  Annual  Performance  Reviews;  3)  Select  and  Support  Artifacts;  4)  Build  the  Portfolio;  5) 
Reflections  Highlight  Professionalism;  and  6)  Polish  and  Package. 

Step  1:  Prepare  for  Success 

The  tried  and  true  advice  we  have  heard  over  the  years  “Think  positive!  Plan  ahead!  Be  prepared!”  is  applicable 
for  those  seeking  tenure.  Essentially,  achieving  tenure  is  an  ongoing  process  that  begins  with  a positive  attitude 
towards  the  task.  Rather  than  viewing  it  as  another  burden,  view  the  process  as  an  opportunity  for  growth  and 
development.  Expect  to  be  successful  and  plan  for  it.  Begin  by  developing  what  we  call  a “working  portfolio.”  A 
working  portfolio  can  be  a physical  or  electronic  folder  where  you  keep  all  materials  relating  to  your  professional 
activities.  Materials  do  not  necessarily  have  to  be  filed  in  any  particular  order,  but  they  must  be  labeled. 

Begin  to  collect  materials  for  your  working  portfolio  in  the  first  year  of  teaching.  Waiting  until  the  year  before  or 
even  the  tenure  review  year  is  a recipe  for  disaster.  To  begin,  use  three  folders  either  paper  or  electronic  to  organize 
materials  that  address  teaching,  scholarship,  and  service.  For  example,  keep  a table  that  charts  courses  taught, 
number  of  students  by  semester  and  maintain  a file  of  your  student  evaluations  in  electronic  or  paper  format.  The 
more  materials  you  are  able  to  accumulate  over  the  course  of  the  five  to  six  years  adds  to  the  variety  and  richness  of 
artifacts  that  you  have  at  your  disposal. 

Most  institutions  have  guidelines  for  tenure  review.  It  is  a good  practice  to  review  those  guidelines  as  early  as 
possible  in  your  career  to  ensure  that  you  understand  tenure  requirements  and  expectations.  The  guidelines  typically 
address  tenure  evaluation  process,  criteria  and  required  components  for  teaching  scholarship,  and  service.  They  may 
also  delineate  specific  levels  of  performance  that  are  acceptable  in  each  category. 

Create  a list  of  the  activities  that  match  each  category  and  get  involved  in  them  as  a participant  and  an  initiator. 
Keep  a record  of  your  participation,  including  a short  description,  date  and  location  of  activities.  Take  pictures 
preferably  in  digital  format  and  scan  or  create  PDF  files  of  all  documents  collected.  Make  short  digital  audio  or 
video  clips  of  your  teaching  or  your  students’  activities.  As  you  gather  students’  products  or  images  remember  to 
obtain  their  permission  to  use  these  artifacts.  Write  a short  reflection  on  the  significance  of  materials  collected.  Keep 
items  your  university  requires  such  as  annual  faculty  reviews  and  student  evaluations. 
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Step  2:  Use  Annual  Performance  Reviews 

You  may  work  for  an  institution  like  ours  that  requires  you  to  prepare  a short  documentation  of  your  professional 
activities  through  the  year  as  part  of  your  yearly  evaluation  review.  In  the  process  of  preparing  the  document,  you 
would  discover  areas  or  categories  in  which  you  have  been  strong  and  those  in  which  you  were  weak.  While 
emphasizing  your  strength,  use  the  review  as  an  opportunity  to  make  plans  to  address  your  areas  of  weakness.  If 
your  annual  review  includes  a self  rating  and  a conversation  with  your  dean  to  discuss  his/her  ratings  of  your 
performance,  use  this  feedback  constructively. 

Students’  perspectives  are  part  of  the  annual  review  and  often  a concern,  especially  if  the  students’  ratings  are  a 
big  factor  in  the  final  evaluations  of  your  teaching  competencies.  Even  though  you  may  perceive  students’  rating  of 
your  teaching  as  incorrect  and  sometimes  unfair,  use  students’  feedback  constructively  to  see  which  teaching 
strategies  worked  and  which  did  not.  From  that  analysis  you  can  identify  areas  which  need  improvement.  After  all, 
isn’t  that  what  teaching  is  all  about? 

Step  3:  Select  and  Support  Artifacts 

From  your  working  portfolio  select  artifacts  that  represent  teaching,  scholarship,  and  service.  You  do  not  have  to 
use  all  artifacts  you  have  collected  over  the  years.  Use  representative  artifacts.  Decisions  about  artifacts  selection 
should  begin  with  a purpose  in  mind.  Each  artifact  must  meet  the  criteria  for  inclusion,  namely,  well-written  or  well- 
produced,  short  (except  for  publications),  and  fit  the  space  capacity  of  your  presenting  medium.  You  may  select 
more  than  one  artifact  to  illustrate  a categoiy.  For  example  in  the  teaching  area,  you  may  select  a course  syllabus  for 
each  course  you  have  taught  in  the  period  under  review  or  syllabi  that  demonstrate  developmental  view  of  your 
course  planning  over  time.  Other  selections  are  best  synthesized  into  charts,  such  as  lists  of  courses  taught, 
enrollment,  and  student  evaluations.  When  using  charts  as  the  organizing  framework,  convert  original  artifacts  into 
PDF  format  and  hyperlink  to  the  chart.  You  may  select  textual  and  image  representations  of  the  same  concept,  or 
samples  of  activities  in  multimedia  format.  When  addressing  publications,  each  must  be  briefly  described  and  the 
full  publication,  preferably  the  published  copy,  should  be  scanned  and  hyperlinked  to  the  points  where  they  are 
referenced.  Community  service  events  in  which  you  participated  are  better  represented  through  pictures  or  videos. 
Students’  portfolios  are  better  documentation  and  illustration  of  the  quality  of  your  teaching  and  students’  learning. 

Selection  of  artifacts  is  the  beginning,  but  the  supporting  explanation  or  reflection  allows  your  audience  to 
understand  your  rationale  for  selecting  a particular  artifact,  as  well  as  the  value  of  the  artifact  (Constantino  & De 
Lorenzo,  2002).  It  is  the  explanation  that  links  the  significance  of  artifacts  in  the  audiences’  mind.  For  example,  one 
author  wanted  to  highlight  her  use  of  graphic  organizers,  so  she  selected  course  graphic  maps,  lecture  material 
graphic  maps,  and  students’  graphic  maps  of  course  content.  Below  is  her  reflective  explanation: 

Graphic  mapping  is  a proven  technique  to  support  learning.  1 use  graphic  maps  to  assist  A&M -Texarkana 
students’  understanding  of  the  overall  organization  and  goals  [hyperlink  to  course  map]  of  a course,  as  well 
as  to  enhance  understanding  of  content  [hyperlink  to  lecture  map].  One  extensive  project  that  1 used  in 
SPED  410  required  that  students  use  Inspiration  software  to  teach  a lesson  [hyperlink  to  students’  maps]  to 
a small  group  covering  specific  course  content.  The  student  Journal  reflections  [hyperlink  to  students’ 
reflections]  addressing  the  Inspiration  project  were  powerful  testimonies  to  student  learning  about 
technology,  course  content,  and  pedagogy  (Harris,  Tenure  Portfolio,  spring  2005). 

It  is  important  to  carefully  consider  the  artifacts  selected  for  your  portfolio;  however  it  is  the  explanations  that 
weave  the  artifacts  together  and  lead  the  audience  to  an  in  depth  understanding  of  your  practice. 

Step  4:  Build  the  Portfolio 

One  of  the  most  important  components  is  the  portfolio  introduction,  which  sets  the  tone  of  the  presentation  of 
your  work.  This  preamble  should  emphasize  the  purpose  and  breadth  of  coverage  of  the  portfolio.  The  preamble  can 
be  part  of  a cover  letter  or  it  can  be  a separate  document.  Use  the  preamble  as  an  opportunity  to  introduce  yourself 
and  emphasize  the  philosophical  or  theoretical  underpinnings  of  your  professional  life.  Here  is  an  extract  from  the 
preamble  of  one  of  the  authors’  tenure  portfolio: 

The  initial  purpose  of  this  portfolio  was  to  provide  evidence  that  I had  met  the tenure  criteria.  As  I 

made  lists  and  gathered  materials,  I found  that  I wanted  to  do  more  than  document  my  accomplishments. 
Simply  put,  I wanted  to  explain  and  describe  “Why  I do  what  I do.”  For  those  reasons,  I chose  to  create  a 
reflective  professional  portfolio  in  an  electronic  format  to  present  myself  and  my  work.  Capturing  the  heart 
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and  soul  of  my  teaching  has  been  a complex  difficult  journey.  Along  the  way  I discovered  my  scholarship, 
renewed  my  dedication  to  service,  and  affirmed  my  commitment  to  teaching.  (Harris,  Tenure  Portfolio, 
spring  2005). 

Following  the  preamble  you  need  an  organizing  framework  to  direct  the  audience  to  the  main  sections  namely, 
Teaching,  Scholarship,  and  Service.  A holistic  view  of  all  sections  may  be  presented  either  as  site  map  or  table  of 
contents.  The  advantage  of  an  electronic  portfolio  is  that  you  can  hyperlink  artifacts  to  the  site  map  or  table  of 
contents.  Figure  1 below  is  an  example  of  a site  map  used  as  an  organizing  framework. 

Figure  1:  Site  Map  (Harris,  Tenure  Portfolio,  spring  2005). 
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At  the  beginning  of  each  section  write  an  introductory  statement.  Use  this  as  an  opportunity  to  highlight  the 
significance  of  each  component. 


Demonstrating  Teaching  Excellence 

Begin  the  presentation  of  teaching  effectiveness  section  with  a philosophy  of  teaching  and  learning.  What 
philosophical  beliefs  shape  course  development  and  implementation?  How  a faculty  translates  these  beliefs  into  the 
structure  of  the  course  syllabus,  development  of  teaching  resources  to  support  learning,  the  implementation  of  the 
course,  and  facilitation  of  students’  learning,  should  be  documented  in  the  portfolio.  In  addition,  provide  hyperlinks 
to  evidence  of  each  for  verification. 

It  is  important  that  faculty  realize  that  a tenure  portfolio  may  be  the  only  place  where  they  can  demonstrate 
evidence  of  effective  teaching  on  their  own  terms.  A portfolio  allows  the  educator  to  “represent  the  complexity  and 
context-dependent  judgment  that  characterizes  teaching”  (Shulman,  1998,  p.31)  and  contains  documentation  of 
teacher  performance  and  student  learning  (Painter  and  Wetzel,  2005).  “Educators  must  demonstrate  that  the  hours 
spent  in  the  classroom  are  only  a part  of  the  real  work  of  teaching.  The  teaching  portfolio  can  provide  professors 
with  a vehicle  to  document  the  quality  and  quantity  of  their  teaching”  (Murray,  1997,  p.  1).  Therefore,  a detailed 
explanation  of  course  development  process  is  necessary,  rather  than  just  a listing  of  course  syllabus  components. 
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“Much  of  the  portfolio  should  be  devoted  to  reflection  on  how  your  actions  are  congruent  with  your  beliefs  about 
teaching.  Most  portfolios  would  also  incorporate  a plan  for  altering  behaviors  found  to  be  incongment  with  the 
philosophical  assumptions  about  teaching  and  learning.  And  the  portfolio  should  incorporate  a strategy  to  assess  the 
appropriateness  and  success  of  the  new  behaviors.”  (Murray,  1997,  p.2). 

We  had  Murray’s  and  others’  advice  at  the  back  of  our  minds  as  we  developed  the  teaching  sections  of  our 
portfolios.  We  began  with  a narrative  of  our  philosophies  of  teaching  and  learning.  We  addressed  the  philosophical 
beliefs  that  shaped  our  course  development  and  delivery.  We  made  apparent  the  process  whereby  we  wove  our 
beliefs  into  the  structure  of  the  entire  course.  Below  is  an  example  of  introduction  to  the  teaching  section  of  one  of 
the  authors’  tenure  portfolio: 

Teaching  has  always  been  my  passion. . .1  grew  up  knowing  the  value  of  a good  education,  and  I have 
strived  to  be  not  just  a good  educator,  but  a creative  one.  A creative  educator  finds  several  ways  of  reaching 
learners.  As  Lee  Shulman  once  said,  if  you  have  150  students  in  your  class,  there  are  "150  Ways  of 
Knowing."  So  one  size  does  not  fit  all,  and  you  must  find  ways  to  reach  all  students.  As  a result,  I never 
teach  the  same  course  the  same  way  twice.  For  each  group  of  students  that  I come  in  contact  with  every 
semester,  I try  to  "listen"  to  their  needs  and  tailor  my  teaching  to  meet  those  needs.  In  the  design  of  my 
courses,  I have  a variety  of  activities  which  provide  students  with  several  opportunities  to  succeed.  There 
are  projects  that  enable  them  to  develop  learning  as  well  as  technology  skills.  At  the  same  time,  there  are 
course  notes  that  enhance  their  knowledge  construction.  I strongly  believe  in  the  Constructivists  view 
which  says  that  learners  are  not  just  empty  vessels  that  you  fill  with  knowledge.  Learners  actively  construct 
their  own  knowledge  from  information  available  to  them.  With  this  in  my  mind,  I try  to  create  learning 
environments  that  will  increase  the  opportunity  for  learners  to  construct  knowledge  in  ways  that  are 
meaningful  to  them.  As  you  will  find  in  my  course  outlines,  examinations  and  tests  are  only  a small 
proportion  of  my  course  evaluation.  To  my  thinking,  examinations  develop  ability  to  recall  information 
from  memory,  whereas  our  students  would  go  into  a world  where  practical  knowledge  and  skills  in 
information  access  and  usage  are  most  valued. . . (Aworuwa,  Tenure  Portfolio,  spring  2005) 

These  philosophical  beliefs  about  teaching  and  learning  influenced  the  way  we  planned  learning  tasks  for  our 
students. 


The  assignment  samples  presented  here  were  created  so  that  one  assignment  is  a precursor  to  another  and 
then  integrated  at  the  final  phase.  In  the  first  assignment  {hyperlink},  students  research  assistive  or  adaptive 
technology  for  persons  with  disability  from  the  Internet.  This  allow  the  students  to  learn  to  use  Internet 
research  tools  as  well  as  learn  about  the  social  and  ethical  issues  of  using  technology  (which  is  one  of  the 
competencies  of  the  TExES)  [Texas  teacher  certification  examination]  [sic]. 

The  second  assignment  requires  students  to  use  their  research  notes  from  previous  assignment  to 
develop  a PowerPoint  presentation  {hyperlinked  evidence}  to  be  delivered  to  an  audience.  This  allows 
students  to  learn  to  use  a technology  tool  to  develop  communication  skills  (another  TExES  competency) 
[sic] 

. . . Each  major  assignment  is  accompanied  by  a rubric  or  checklist  that  informs  students  about  how 
they  will  be  graded  on  the  assignment  and  also  serves  an  advance  organizer  of  the  quality  of  work  expected 
from  the  students.  (Aworuwa,  Tenure  Portfolio,  spring  2005). 

In  cases  when  teaching  actions  have  unintended  consequences,  it  is  important  to  reflect  on  why,  make  corrective 
actions  and  analyze  the  impact  on  your  teaching  and  students’  learning.  Such  instances  provide  opportunity  for 
growth  by  deepening  our  understanding  of  who  we  are  as  teachers  and  who  our  students  are  as  learners.  As  one  of 
the  authors  reflected  on  a course  she  has  taught  for  several  semesters,  she  discovered  several  misconceptions  both 
she  and  her  students  brought  into  the  course. 

When  I first  taught  this  course  in  the  summer  of  2002,  there  were  misconceptions  on  my  part  and  on  the 
part  of  the  students  . . .First,  I assumed  my  students  were  all  education  majors,  so  I used  a lot  of  education 
concepts  and  examples  in  the  course.  I soon  realized  that  I had  students  taking  the  class  who  were  other  majors 
including  psychology  and  criminal  justice.  Even  ‘education’  students  were  not  familiar  with  the  education 
concepts  and  terminologies  I was  using. . . [because  this  course  was  taken  prior  to  foundational  education 
courses]. 
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The  misconception  on  the  part  of  my  students  included  an  assumption  that  the  class  was  just  like  any  other 
class  in  which  they  have  specific  chapters  to  read  and  they  take  a test  and  it  is  over.  They  were  not  prepared  for 
the  amount  of  new  and  technical  concepts  and  the  time  commitment.  For  many  of  them,  the  course  was  the  first 
time  they  would  learn  about  computers.  The  situation  was  not  helped  by  the  fact  that  there  was  no  textbook  . . . 
Flow  do  you  select  a textbook  on  computer  technologies  that  will  meet  the  needs  of  all  the  majors. . . [even 
electronic  notes  were  a source  of  confusion  because  students  failed  to  connect  to  hyperlinked  materials]. 

. ..,  I gradually  developed  the  course  notes  and  tutorials. . .into  a tutorial  book  (see:  As  easy  as  1-2-3:  A 
tutorial  on  basic  computer  skills  for  teachers)  {hyperlinked  document}.  The  course  had  better  students’  ratings 
the  semester  we  first  used  the  tutorial  book. . . 

Other  improvements  that  I have  made  to  the  course  include  streamlining  assignments  so  that  students  can 
complete  a sizeable  portion  during  class  through  hands-on  activities  (Aworuwa,  Tenure  Portfolio,  spring  2005). 

Throughout  the  teaching  section  highlight  instructional  strategies  that  you  are  excited  about  and  which  result  in 
profound  learning  experiences  for  your  students.  Facilitate  the  audience’s  understanding  of  your  teaching 
enthusiasm  and  student’s  learning  through  reflective  explanations. 

Demonstrating  Excellence  in  Scholarship 

Within  the  scholarship  section  it  is  important  to  describe  how  you  have  met  the  criteria  specified  in  your 
university’s  guidelines.  Flere  is  how  one  of  the  authors  presented  evidence  demonstrating  how  she  met  the 
scholarship  requirements: 

Evidence  that  I have  met  and  surpassed  [university’s]  tenure  criteria  for  scholarship  includes:  (a) 
membership  in  the  Academy  for  Educator  Development;  (b)  six  collaborative  grants  totally  $45,150;  (c)  14 
presentations  at  national  juried  conferences  and  numerous  state,  regional,  and  local  presentations  and 
workshops;  and  (d)  two  publications,  one  review  of  a textbook  for  Allyn  and  Bacon  publishers,  and  a 
journal  article  submitted  to  the  Journal  of  Excellence  in  College  Teaching.  Most  significant  for  me  on  a 
professional  level  is  not  the  count  of  scholarship  activities  but  the  manner  in  which  scholarship  enriches  my 
teaching  and  vice  versa.  It  is  a revelation  for  me  that  the  scholarship  and  teaching  are  so  interwoven  each 
shaping  and  enriching  the  other  (Flarris,  Tenure  Portfolio,  spring  2005). 

The  reflective  introductions  depend  on  your  style.  Here  is  another  approach. 

Scholarship  is  a part  of  my  professional  life  that  I realize  I now  enjoy  as  much  as  I enjoy  teaching. 
Through  the  Regents'  Initiative  fund,  I was  privileged  to  attend  a "Writing  for  Professional  Publication" 
workshop  conducted  by  ...  .1  learned  from  the  workshop  how  to  link  research  and  teaching  so  that  my 
research  informs  my  teaching  and  vice  versa.  In  addition,  being  a member  of  the  Academy  for  Educator 
Development  has  had  a positive  impact  on  my  scholarship.  It  provided  me  the  opportunity  to  attend  several 
workshops  and  conferences,  form  important  alliances,  and  enhance  my  professional  development. 

I have  listed  in  my  resume  several  scholarly  activities  in  which  I was  engaged  during  the  period 
under  review.  They  included  research  and  publications,  grants,  conferences,  workshops  attended  for  my 
own  professional  development,  and  workshops  that  I have  conducted  for  others.  (Aworuwa,  Tenure 
Portfolio,  spring  2005). 

Group  the  different  types  of  scholarly  activities  such  as  grants,  publications,  conference  presentations,  professional 
development,  research  agenda,  and  professional  organizations  and  hyperlink  to  artifacts  selected  for  each  group.  If 
you  find  gaps  in  meeting  the  university’s  requirements  provide  an  explanation.  See  an  example: 

I have  not  been  able  to  achieve  much  in  the  area  of  grants  in  the  last  two  years.  I hope  to  do  more  when  I 
complete  the  development  of  my  courses. . .The  grants  information  presented  relate  to  my  activities  with  the 
Academy  for  Educator  Development  Fellowship  grant,  MasTAR  - an  unfunded  grant,  and  previous  grant 
project  that  I worked  on  shortly  before  1 joined  the  faculty.  ..(Aworawa,  Tenure  Portfolio,  spring  2005). 

The  significance  of  the  scholarship  section  is  critical  tenure  component;  therefore  try  to  represent  as  many  areas  of 
scholarship  as  possible. 
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Demonstrating  Excellence  in  Service 

As  in  your  Teaching  and  Scholarship  sections,  your  Service  area  also  needs  a quality  introduction.  For  example: 

According  to  the  College's  guidelines  for  Tenure  and  Promotion,  faculty  members  are  expected  to  be 
actively  engaged  in  the  academic  governance  of  the  University,  College,  and  Department.  Faculty  members 
are  also  expected  to  actively  participate  in  activities  of  their  profession,  as  well  as  provide  service  to  the 
community.  I believe  I have  met  and  exceeded  expectations  in  all  service  categories  - namely,  university, 
profession,  and  the  community.  Service  has  always  been  a part  of  my  life.  I believe  in  giving  one  hundred 
percent  in  all  that  1 do,  and  service  to  my  fellow  human  beings  is  one  of  the  highest  callings  a person  can 
receive.  Even  though  I may  catalog  a list  of  activities  in  which  I had  been  involved,  I still  believe  that  I 
have  not  done  enough  to  serve  my  community.  I have  always  felt  that  way,  and  I think  I will  continue  to 
feel  that  way.  Maybe  we  all  need  to  feel  we  have  not  done  enough  to  keep  us  on  our  toes  (Aworuwa, 

Tenure  Portfolio,  spring  2005).. 

Think  about  the  different  categories  found  within  the  service  area  and  present  a reflective  summary  of  your 
contributions  and  hyperlink  each  to  the  artifacts.  FI  ere  is  an  example  of  how  one  of  the  authors  presented  her  service 
to  her  profession: 

I affiliate  with  a number  of  professional  organizations  for  the  purpose  of  promoting  the  missions  of  the 
organizations,  as  well  as  keeping  me  abreast  of  developments  in  my  fields  and  for  professional  growth.  I 
have  been  more  active  in  some  of  the  professional  organizations  and  divisions  than  I have  been  in  others.  1 
had  served  as  a conference  proposal  reviewer  [hyperlink  to  email  assigning  reviews]  for  AECT  divisions 
since  2002,  and  as  a conference  session  facilitator  [hyperlink  to  email  documenting  session  facilitator 
assignments]  for  several  sessions  at  the  AECT  Conference.  I have  also  been  more  active  with  some  of  the 
listserv  and  weblog  activities  of  the  Distance  Learning  division,  since  its  objectives  are  more  in  line  with 
what  I currently  do  in  my  job.  I served  as  an  expert  panelist  for  an  AECT/PT3  Catalyst  project  [hyperlink 
to  letter]  ....  The  purpose  of  the  project  was  to  identify  technology -based  teaching  strategies  of  various 
education  professionals  in  order  to  develop  certificate  programs  for  them.  Expert  panelists  helped  to 
identify  the  strategies  (Aworuwa,  Tenure  Portfolio,  spring  2005). 

For  community  service  often  photographs  tell  the  story  best.  Photographs  may  be  hyperlinked  or  presented  as  an 
electronic  album.  Sometimes  you  do  not  necessarily  have  to  have  a leadership  role  for  your  service  to  be  significant: 

My  part  in  [name  of  local  children’s  park]  was  very  small,  but  as  . . . would  say  every  part  counts!  My 
interest  was  to  get  the  . . . students  involved.  For  the  design  day  with  hundreds  of  school  kids  at  [location  of 
event],  1 enlisted. . . education  students  as  volunteers.  I was  so  inspired  by  the  day  I wrote  Why  Build  a 
Park?  [hyperlink  to  essay].  Much  to  my  surprise  and  pleasure,  [the  English  professor]  thought  my  essay 
would  inspire  [university’s]  faculty  and  staff  to  make  donations,  so  she  sent  it  out  (Harris,  Tenure  Portfolio, 
spring  2005). 

Service  is  an  area  in  which  each  faculty  member  can  demonstrate  contributions  and  their  individualism.  This  section 
may  have  a completely  different  tone  than  Teaching  or  Scholarship  sections  which  adds  depth  and  breadth  to  your 
portfolio. 

Other  Components  and  Required  Documentations 

Within  the  tenure  review  process  your  institution  may  require  specific  components  such  as  an  application  letter, 
vita,  annual  evaluations,  lists  of  courses  per  semester  with  enrollment,  and  student  evaluations. 

The  placement  of  these  items  in  your  portfolio  depends  on  your  style  and  the  emphasis  you  want  to  create.  For 
example,  the  letter  of  application  may  be  placed  as  the  first  page  followed  by  a table  of  contents  which  hyperlinks  to 
your  vita,  evaluations  materials,  followed  by  Teaching,  Scholarship,  and  Service  sections.  To  address  evaluation  one 
of  the  authors  created  an  evaluation  section,  placed  after  her  vita.  She  used  a table  summarizing  annual  faculty 
performance  evaluations  for  each  year,  hyperlinked  to  a copy  of  the  evaluation  documents. 

Other  materials  included  were  a pre-tenure  review  report,  information  about  promotion,  and  student  evaluations. 
The  other  author,  on  the  other  hand,  addressed  evaluation  under  two  sections.  The  annual  faculty  performance 
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evaluation  was  treated  in  a separate  section  while  student  evaluation  and  course  were  addressed  as  part  of  teaching 
effectiveness  section. 

Finally,  it  is  important  to  note  that  you  may  create  sections  to  address  accomplishments  that  do  not  fit  in  the 
standard  sections.  For  example  one  of  the  authors  included  an  “Other”  section  in  which  she  presented  awards  and 
letters  of  appreciation. 

Step  5:  Reflections  Highlight  Professionalism 

Throughout  this  article  we  have  embedded  examples  of  the  reflective  explanations  that  support  artifacts. 
Reflection  weaves  together  each  detail  of  your  portfolio  and  creates  a holistic  understanding  of  depth  and  breadth  of 
your  work.  The  point  at  which  you  begin  the  reflection  part  of  your  portfolio  is  a stylistic  choice.  You  may  write  the 
reflections  first,  last,  or  even  during  the  middle  of  the  portfolio  process.  Each  approach  is  valid.  What  is  important  is 
emphasizing  how  each  artifact  demonstrates  continuous  growth  and  development  of  your  teaching  knowledge  and 
practices,  and  how  it  demonstrates  the  philosophy  that  informs  your  professional  endeavors.  One  author  actually 
used  the  reflective  introductions  to  each  section.  Teaching,  Scholarship,  and  Service  as  the  core  of  her  application 
letter.  The  other  author  concluded  the  portfolio  with  a final  reflection. 

Step  6:  Polish  and  Package 

Presentation  matters.  Polish  and  package  the  portfolio  before  presenting  it  to  the  intended  audience.  The  table  of 
content,  overview  of  the  portfolio,  and  conclusion  to  the  portfolio  contribute  to  the  flow  of  the  materials.  You  may 
construct  the  table  of  contents  as  you  develop  the  portfolio,  or  complete  it  after  all  sections  have  been  constructed. 
Once  again,  this  is  a stylistic  choice  and  both  approaches  are  appropriate.  Each  section  may  have  a table  of  contents 
as  well.  Hyperlinks  from  these  organizing  frameworks  allow  for  multiple  points  of  access  to  your  information.  Be 
cautious  about  the  number  of  layers  you  create  with  your  hyperlinks,  because  some  audiences  dislike  more  than 
three  or  four  layers. 

Use  color  to  enhance  the  visual  quality  of  your  portfolio.  Soothing  and  attractive  colors  prevent  your  portfolio 
from  becoming  boring  and  monotonous.  Colors  that  are  too  bright  or  too  dark  may  discourage  your  audience  rather 
than  engage  them.  Pages  with  related  information  may  have  the  same  color  scheme,  to  facilitate  ease  of  transition. 
Font  color  and  size  are  also  important  to  enhance  your  presentation.  Finally,  there  are  several  options  of  submitting 
your  electronic  portfolio.  You  may  send  it  electronically  or  save  to  a CD  or  DVD  with  a cover  letter  in  a 
presentation  binder. 


Conclusion 

Creating  a tenure-winning  portfolio  starts  at  the  beginning  of  your  teaching  career.  It  can  be  an  arduous  task  or  a 
rewarding  experience,  depending  on  your  attitude  and  level  of  preparation.  To  the  authors,  it  was  a revealing 
experience,  an  opportunity  to  learn  about  ourselves  and  our  career.  Constructing  our  portfolios  in  electronic  format 
provided  us  the  flexibility  to  show  the  richness  and  diversified  experiences  of  our  professional  lives. 
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Introduction: 


Podcasting  as  on  Demand  Educational  Support 

Keith  R.  Houk,  M.F.A. 

University  of  Houston  school  of  Communication 


Podcasting  in  all  its  various  permutations  of  audio,  enhanced  and  vodcast,  represents  the  newest  form  of  delivery  for 
educational  content  to  students.  The  number  of  colleges  and  universities  jumping  on  the  Podcasting  bandwagon 
grows  each  day  with  new  methods  of  instructional  implementation  being  rapidly  developed.  “Podcasting"  derives  its 
name  from  iPod  (Apple’s  seemingly  omnipresent  mobile  media  device  or  MMD  for  short)  and  “broadcasting”.  A 
podcast  is  a highly  compressed  file  of  pre-recorded  material.  Podcast  can  be  audio  only,  or  they  can  include  visual 
elements  as  well.  They  are  relatively  easy  to  create  and  extremely  easy  to  disseminate  to  an  audience.  They  are  not 
live  broadcasts,  but  rather  stored  files  allowing  for  asynchronous  access  by  an  audience.  In  reality,  a podcast  is 
simply  a dressed  up  MP3  file  with  a new  name  and  a lot  of  pop  culture  buzz  surrounding  it.  The  thing  that  makes  a 
podcast  stand  out  from  other  audio  files  floating  around  in  cyberspace  is  the  ability  to  subscribe  to  the  material. 
Updates  and  new  episodes  of  podcast  material  are  just  a subscribe  button  away,  meaning  that  audiences  can  stay  up 
to  date  without  having  to  constantly  check  a website  for  new  material.  Proponents  of  using  the  technology  in  an 
educational  environment  often  reference  podcasting  as  if  it  is  some  magical  elixir,  a panacea  for  educators 
everywhere.  Podcast  your  courses,  that’ll  fix  ’em,  students  will  learn,  graduation  rates  will  skyrocket  and  all  will  be 
well  with  the  educational  world.  Often  times,  podcasting  detractors  express  feelings  ranging  from  skepticism  to 
near  outrage  with  protestations  of  “fad”  and  modern  day  educational  snake  oil.  Students  will  quit  coming  to  class, 
it’s  a gimmick,  a simulacrum  of  the  true  classroom  experience,  the  lecture  hall’s  doppleganger.  Indeed,  both  sides  of 
the  aisle  might  have  a point  with  their  statements.  The  simple  fact  of  the  matter  is  that  “podcasting”  is  a relatively 
new  technology  with  multiple  potential  applications  that  are  still  being  explored  and  studied.  While  anecdotal 
information  on  the  effectiveness  of  podcasting  is  abundant,  few  studies  have  been  conducted  to  quantitatively  gauge 
its  impact  on  the  educational  process.  The  School  of  Communication  at  the  University  of  Houston  has  been  engaged 
in  such  a study  since  the  beginning  of  the  Fall  semester  of  2005,  with  initial  data  collected  during  the  Spring 
semester  of  2006.  The  findings  of  the  initial  study  give  credence  to  some  of  the  opinions  of  the  proponents  as  well  as 
opponents  of  “podcasting”. (University  of  Houston,  2006)  The  data  collected  by  the  UH  study  demonstrates  some 
interesting  patterns  in  student  utilization  of  podcast  course  material  and  clarifies  some  of  what  works  and  what  does 
not  in  academic  podcasting. 

Overall,  the  study  indicates  the  potential  for  podcasts  to  be  used  very  effectively  as  on  demand  educational  support. 
This  on  demand  support  might  take  the  form  of  an  audio  recording  of  a lecture  that  a student  uses  to  review  for  a 
test,  but  it  can  be  much  more  than  a mere  copy  of  the  class  room  experience.  This  paper  proposes  an  argument  for 
the  inclusion  of  podcasting  in  the  college  classroom,  the  pedagogical  efficacy  of  such  an  implementation,  and  offer 
some  suggestions  towards  making  the  most  out  of  educational  podcasting.  This  argument  is  supported  by  research 
data  gathered  by  the  University  of  Houston  Podcasting  pilot  program. 

The  Study 

The  University  of  Houston  Podcasting  Pilot  program  was  first  implemented  in  the  Spring  of  2006.  Two  courses 
were  involved  in  the  first  round  of  the  study.  Four  sections  were  involved,  three  lecturing  professors,  one  online 
section  and  one  early  morning  (8  a.m.)  section.  In  total,  the  survey  population  included  200  potential  participants 
with  165  actual  respondents  and  a gender  breakdown  of  60%  female  to  40%  male.  The  survey  instrument  consisted 
of  74  questions  spread  over  seven  sections  of  varying  length.  The  majority  (57%)  of  students  were  aware  of  what  a 
podcast  was  prior  to  taking  the  course,  but  less  than  a third  (31%)  had  actually  listened  to  a podcast  before.  That  is 
extremely  important  to  note  as  an  educator.  Podcasting  is  definitely  a buzzword  in  academics  right  now,  but  before 
faculty  rush  head  first  into  content  delivery,  they  should  carefully  gauge  the  student’s  understanding,  ability  and 
willingness  to  interact  with  the  new  technology. 

The  necessary  software  for  downloading  and  subscribing  to  podcasts  is  available  as  free  downloads.  A student  does 
not  need  to  own  a MMD  to  be  able  to  receive  content,  but  87%  of  the  UH  sample  population  reported  owning  at 
least  one  MMD.  The  quality  of  the  delivered  material  with  all  three  types  of  podcasts  (audio  only,  enhanced  with 
still  images  and  video)  received  over  75%  good  or  excellent  ratings.  There  was  no  clear-cut  favorite  among  the 
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three  possible  delivery  methods.  The  population  from  the  four  survey  courses  indicated  a very  slight  preference  for 
video  podcasts  over  enhanced  and  audio  only  podcasts.  (Figure  1) 


A multiple  regression  analysis  performed  on  the  UH  data  determine  that  students  who  were  “more  likely  to  view  the 
addition  of  podcasting  material  as  helpful  to  the  improvement  of  their  learning  outcome”  had  the  following 
characteristics: 

•They  were  younger; 

•Had  better  quality  of  MMD’s; 

•Had  a higher  ability  to  retain  information  while  using  MMD  to  access  instructional  materials  while  involved  in 
other  activities; 

•Had  viewed  podcasting  course  materials  on  a computer; 

•Had  not  experienced  technical  issues  related  to  the  use  of  MMD’s; 

•Believed  that  an  e-mail  reminder  would  be  useful  whenever  a new  podcast  was  available. 

The  analysis  also  determined  that  gender,  length  of  time  using  computers  (years  of  experienced),  commute  time, 
type  of  internet  connection,  length  of  time  using  MMD’s  to  access  course  material  and  viewing  material  on  an 
MMD  versus  a computer  were  not  statistically  significant  factors.  It  is  important  to  note  that  the  University  of 
Houston  is  located  in  a sprawling  community  with  a very  geographically  diverse  student  population.  Over  48%  of 
those  surveyed  had  at  least  a half  hour  commute  to  campus,  with  nearly  a quarter  spending  over  an  hour  getting  to 
class  each  day.  Although  commute  times  did  not  apparently  play  a factor  in  student’s  opinion  of  podcasts,  the  UH 
study  should  serve  as  a good  guide  for  large  urban  institutions,  but  might  be  slightly  out  of  balance  for  more 
traditional  “college  town”  campuses  with  students  residing  on  or  in  the  nearby  vicinity  of  the  institution  (University 
of  Houston,  2006) 

Podcasting  as  On  Demand  Educational  Support: 

To  return  to  the  argument  in  favor  of  using  podcasts  as  supplemental  and/or  more  detailed  explanatory  materials, 
take  for  instance  the  potential  applications  in  Introductory  and  Survey  classes.  Such  courses  often  pose  a variety  of 
problems  for  students  as  well  as  professors.  The  classes  generally  fulfill  a core  curriculum  requirement  for 
universities  and  will  be  populated  by  students  with  varying  degrees  of  interest  in  and  knowledge  of  the  subject 
matter.  Professors  are  required  to  cover  vast  amounts  of  material,  which  may  or  may  not  be  in  their  particular  area 
of  expertise.  Lecture  time  is  limited,  as  is  the  time  for  classroom  Q & A.  Potential  for  one-on-one  communication 
with  every  student  is  little  more  than  a pipe  dream.  A history  course  covering  300  years  cannot  spend  three  weeks 
discussing  one  eventful  year  and  still  hope  to  cover  the  remaining  299  years  with  any  degree  of  success.  Students 
who  find  themselves  falling  behind  are  left  to  re-read  notes,  confer  with  classmates,  hire  a tutor  or  possibly  catch 
their  professor  for  a review  session.  Similarly,  students  with  a great  interest  in  the  material  are  left  wanting  more. 

By  carefully  identifying  the  muddiest  topics  that  arise  during  the  course  of  a semester,  an  instructor  can  pre -produce 
discussions  dealing  with  those  difficult  those  areas.  These  discussions  can  take  the  form  of  audio  or  video  podcasts 
and  serve  as  on  demand  “tutoring”  sessions  for  students  who  find  themselves  falling  behind  in  class.  The  content  of 
the  discussions  would  in  part  dictate  the  need  for  video  content  as  opposed  to  audio  only.  Remember  that  engaging 
a video  podcast  requires  a different  level  of  interactivity  with  a MMD.  Offering  video  only  versions  of  course 
material  limits  the  areas  and  activities  in  which  a student  can  engage  with  the  material,  requires  larger  files  with 
longer  download  times  and  requires  more  storage  space  on  servers  as  well  as  students’  MMDs.  Video  podcast  also 
reduce  the  chance  for  students  who  drive  to  school  to  access  the  material  during  a lengthy  commute.  No  one  wants 
students  to  drive  while  trying  to  watch  lecture  material  on  a 2.5-inch  screen.  Muddy  areas  can  result  from  a variety 
of  reasons  and  should  be  easy  to  identify  from  student  feedback  and  from  test  results  from  prior  semesters. 
Supplemental  podcasts  can  be  produced  before  a semester  begins  based  on  prior  test  results.  If  there  is  an  area  that 
each  semester  consistently  causes  a large  number  of  people  difficulty,  simply  pre-record  a podcast  before  the 
semester  begins.  This  podcast  can  then  be  made  available  to  anyone  wishing  clarification  of  subject  material.  It 
might  be  more  detailed  than  the  original  lecture  or  it  might  offer  differing  examples  than  in  class  discussion.  Linda 
Herkenhoff,  an  assistant  professor  in  the  graduate  department  at  St.  Mary’s  College  creates  two  podcast  summaries 
of  her  lectures.  One  is  designed  for  students  lagging  behind  and  in  need  of  extra  explanation  for  tricky  concepts.  The 
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other  is  for  advanced  students  wanting  more  detail  from  the  lectures.  (Read,  2006)  Supplemental  material  can  also 
be  fairly  easily  created  during  the  semester  based  on  problems  cropping  up  during  class. 

Students  are  often  performing  a delicate  balancing  act  throughout  the  semester.  Classes,  jobs  and  personal  life  can 
collide  into  each  other  from  time  to  time.  A student  might  be  in  class  taking  notes,  seeming  to  pay  attention,  all  the 
while  thinking  about  another  course,  a bad  day  at  a part  time  job  or  a relationship  that  is  falling  apart.  These 
problems  do  arise,  and  one  inattentive  day  or  a day  missed  for  illness  or  personal  reasons  can  negatively  impact  a 
semester.  Providing  podcasts  of  lectures  can  prove  to  be  valuable  for  students  who  might  have  otherwise  fallen 
behind.  Podcast  lectures  can  also  be  advantageous  to  students  who  wish  to  preview  lecture  material.  A student  can 
listen  to  course  material  the  night  before  a lecture  or  even  while  driving  to  class.  Having  heard  the  material 
beforehand,  the  student  can  have  a different  classroom  experience,  one  where  their  time  is  not  spent  struggling  to 
take  notes  and  keep  up,  but  rather  one  where  they  are  actively  engaged  in  the  lecture  - listening  and  absorbing 
material  rather  than  simply  transcribing.  The  University  of  Houston  implemented  podcasts  as  optional  material,  and 
any  previewing  of  lectures  before  class  time  was  strictly  the  students’  choice.  There  were  no  data  collected  on  using 
podcasts  as  a required  portion  of  the  course  materials.  Future  research  could  be  done  to  gauge  the  effectiveness  of 
optional  versus  required  podcasts.  (University  of  Houston,  2006) 

Another  benefit  of  offering  supplemental  on  demand  material  is  the  ability  to  clarify  and  improve  on  any  given 
lecture.  Faculty,  like  students,  can  have  off  days.  If  a class  lecture  didn’t  go  as  smoothly  as  desired,  an  important 
point  wasn’t  quite  as  clear  as  it  needed  to  be,  or  the  lecture  just  didn’t  meet  with  a professor’s  high  standards;  it  is 
possible  to  correct  a deficient  lecture.  A professor  can,  with  the  lecture  still  fresh  in  their  minds,  return  to  their 
office  and  record  an  addendum  to  the  day’s  work,  post  it  online,  let  students  know  it  is  there  with  no  need  to  spend 
the  next  class  period  clarifying  the  previous  lecture  and  losing  valuable  class  time.  Questions  can  also  be  answered 
in  greater  detail  than  class  time  would  allow.  When  an  “I’ll  get  back  to  you  on  that,”  or  “I’ll  need  to  check  the  data” 
question  arises  in  class,  podcasting  makes  it  possible  to  actually  check  the  data  and  get  back  to  the  class.  Again,  this 
can  be  done  without  losing  time  in  the  next  lecture. 

The  supplemental  sessions  can  also  be  designed  to  offer  a broader  explanation  of  a topic  rather  than  simple  remedial 
lectures.  Students  who  have  a greater  interest  in  a topic  and  a desire  to  learn  more  than  the  amount  required  could 
benefit  from  these  sessions  as  well.  As  in  the  case  of  Gardner  Campbell,  Professor  of  English  at  the  University  of 
Mary  Washington,  who  spent  a summer  reading  a John  Donne  poem  a day.  These  readings,  accompanied  by 
Campbell’s  own  commentary  provided  five  to  eight  minute  chunks  of  information  that  “students  can  use  to  help 
them  prepare  for  each  day’s  reading  assignment.”(Campbell,  2005).  By  including  interviews  with  other 
academicians  and  members  of  the  professional  community  students  will  gain  an  understanding  of  the  material  and  a 
broader  range  of  viewpoints  than  simply  that  of  the  lecturer  or  textbook  author.  For  example:  a book  publisher 
could  be  interviewed  to  explain  how  a manuscript  makes  it  to  print;  a television  engineer  could  explain  how  light  is 
converted  to  electrical  information;  a medieval  historian  could  explain  the  impact  of  the  printing  press.  In  an  ideal 
world,  such  topical  interviews  and  mini  lectures  could  be  made  available  on  an  interdisciplinary  scale.  An  interview 
with  a philosophy  professor  on  the  theories  of  Immanuel  Kant  could  be  of  benefit  in  a Communications  class 
discussing  Media  Ethics  or  in  a History  class  discussing  The  Enlightenment.  If  an  institution  allows  for  such 
interdisciplinary  action,  a database  of  podcast  offerings  could  easily  be  built  over  the  course  of  a few  semesters  and 
shared  in  a campus  wide  effort.  At  the  University  of  Denver,  Dr.  Don  McCubbrey  sees  even  greater  potential, 
“Imagine  if  a class  in  Nairobi  could  listen  to  podcasts  from  other  countries.  Podcasts  can  democratize  learning.” 
(Lum,  2006)  For  many,  simply  getting  the  Chemistry  department  to  talk  to  the  Anthropology  department  might  be  a 
big  enough  challenge  to  begin  with.  Still,  the  potential  is  there. 

Regardless  of  the  form  podcasts  take,  one  thing  to  always  remember  is  that  they  should  be  an  educational  value 
added  endeavor,  offering  students  something  more  than  has  been  traditionally  available.  They  do  not  need  to  be  slick 
Hollywood  quality  productions  as  long  as  they  are  somewhat  engaging,  cleanly  recorded  and  informative. 
Remember,  the  audience  for  an  educational  podcast  is  different  than  the  audience  for  an  entertainment  podcast. 

Based  on  student  perception,  podcasts  are  a valuable  tool  in  the  educational  process.  As  shown  in  Figure  2,  67%  of 
the  UH  survey  population  felt  that  the  addition  of  podcasts  was  helpful  in  understanding  course  content  with  59% 
indicating  that  podcasts  helped  them  better  retain  information.  (University  of  Houston,  2006)  A similar  podcasting 
study  conducted  at  the  University  of  Michigan  School  of  Dentistry  found  that  72.7%  of  respondents  felt  that 
including  a podcasting  feed  in  the  class  structure  helped  their  grades.  (Brittain,  et  ah,  2006) 
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Despite  the  great  potential  in  podcasting,  educators  must  be  careful  not  to  overuse  the  new  technology.  While  it  is 
possible  for  students  to  take  their  school  work  with  them  into  their  everyday  lives,  there  is  no  guarantee  that  students 
will  fully  utilize  the  new  technology  in  the  manner  that  educators  desire.  Podcasting  should  not  be  used  in  place  of  a 
traditional  classroom  experience,  but  rather  a supplement  to  the  classroom.  As  the  UH  study  indicates,  the  majority 
of  students  (66%)  favored  a traditional  face-to-face  lecture  experience  with  a podcast  component  to  a podcast  only 
class  format.  (Figure  4) 

The  survey  population  in  the  UF1  study  also  demonstrated  a definite  preference  for  using  their  MMD  to  listen  to 
music  over  course  material.  In  fact,  69%  of  respondents  listed  “listening  to  music  as  the  activity  they  engaged  in 
most  often  with  their  MMD.  Only  1%  of  the  population  indicated  that  “reviewing  course  material'’  was  the  primary 
reason  they  used  their  MMD.  The  mechanism  most  utilized  by  students  to  access  course  materials  was  the  computer. 
63%  of  students  preferred  accessing  podcast  material  on  the  computer  as  opposed  to  31%  who  preferred  to  use  their 
MMD  for  course  review.  There  could  be  a variety  of  reasons  for  students  sticking  with  the  computer.  Storage  space 
is  often  an  issue,  and  given  the  choice  of  a professor’s  lecture  or  an  album’s  worth  of  songs,  the  lecture  is  going  to 
be  left  out  of  the  mix.  There  is  also  the  possibility  that  students  are  unaware  of  the  freedom  of  mobility  that  podcasts 
give  them.  Some  students  immediately  see  the  potential  in  a mobile  classroom  experience,  but  many  do  not.  As  the 
UFI  study  found,  students  who  accessed  the  course  material  via  a computer  were  more  likely  to  recognize  the  value 
of  a podcast  than  those  using  only  MMD’s.  (University  of  Flouston,  2006) 

Questions  of  the  technology’s  ability  to  transfer  the  information  should  not  be  a concern.  The  UH  survey  found  the 
delivery  mechanism  to  be  solid  and  reliable.  A simple  web  page  of  instructions  for  students  can  cover  the  majority 
of  issues  that  do  arise.  The  survey  population  ranked  the  quality  of  the  podcasts  very  highly  both  from  a technical 
standpoint  and  a content  offered  standpoint.  For  content  creation,  the  technology  to  produce  high  quality  podcasts  is 
relatively  inexpensive  and  amounts  to  an  initial  investment  in  basic  equipment  and  software.  Note  that  the  entry 
cost  of  to  podcast  is  minimal,  but  the  time  involved  in  creating  content  can  be  daunting  depending  on  course  needs. 
Determining  the  course  needs  should  be  a careful  and  deliberate  process.  The  choice  of  audio  only,  enhanced 
podcast  or  video  podcast  should  be  made  after  considering  various  questions.  Video  podcasts  will  take  longer  to 
produce,  edit,  upload  and  download.  Does  the  value  added  by  the  video  justify  the  added  time  and  effort?  When 
considering  that  question,  an  instructor  should  also  take  into  account  student  usage  and  preference  of  podcast  types. 
57%  of  the  UH  survey  population  responded  that  video  podcasts  most  suited  their  learning  style,  but  only  35%  of 
respondents  utilized  video  podcasts,  with  34%  favoring  audio  only  and  31%  preferring  enhanced  podcasts.  This 
would  hint  at  students’  choosing  smaller,  more  easily  downloaded  files  over  files  that  they  felt  most  suited  their 
learning  style.  Video  podcasts  might  best  suit  a student’s  learning  style,  but  if  the  video  is  not  adding  more  value 
than  the  audio  only  or  enhanced  podcasts,  students  will  opt  for  something  other  than  video.  The  podcasting  study 
conducted  at  the  University  of  Michigan  School  of  Dentistry  was  initially  prompted  by  the  students’  desire  to  have 
videotaped  lectures  made  available.  In  order  to  determine  the  most  effective  delivery  mode,  video,  enhanced  and 
audio  only  podcasts  were  created.  The  study  found  that,  “Responding  users  both  preferred  (66.1  percent)  and  used 
(66.1  percent)  audio-only  over  both  the  requested  format  of  video  and  PowerPoint  synced  with  audio.”  (Brittain,  et 
ah,  2006) 

It  would  be  a tremendous  oversight  to  ignore  one  of  the  biggest  concerns  lecturers  have  with  this  entire  process: 
attendance.  The  UH  study  shows  that  these  concerns  are  sometimes  valid.  In  many  ways,  the  subject  matter  of  the 
class  and  the  time  of  the  class  play  the  biggest  role  in  determining  if  students  stop  showing  up  for  lectures.  In  the 
research  gathered  by  UH,  one  course  was  held  at  8 a.m.,  and  it  was  not  surprising  to  see  a tremendous  drop  in 
attendance  with  the  introduction  of  podcasts.  In  that  particular  section,  for  research  purposes,  podcasts  were  not 
made  available  until  the  semester  midterm.  At  that  point,  the  instructor  began  recording  in-class  lectures  and 
distributing  them  as  podcasts.  Students  did  stop  coming  to  class,  instead  relying  on  the  podcasts  to  get  them 
through.  It  worked  for  the  students  and  there  was  no  substantial  drop  in  grades  with  the  drop  in  attendance.  The  8 
a.m.  lecture  period  is  an  extreme  example  of  introducing  an  alternative  information  delivery  mechanism  and  having 
a drop  in  attendance.  Other  lecture  times  may  not  be  as  susceptible  to  mass  migrations  to  podcasts.  Students  will 
often  enroll  in  classes  because  they  want  the  personal  face-to-face  offered  by  the  traditional  lecture  model.  In  the 
UH  study,  two  sections  of  a web  design  course  were  offered.  One  was  a traditional  face  to  face  lecture,  the  other 
was  an  online  Distance  Ed  class.  The  students  in  the  online  class  preferred  receiving  podcasts  to  attending  a 
traditional  lecture.  The  students  that  enrolled  in  the  face  to  face  lecture  preferred  the  classroom  to  the  podcast.  The 
subject  matter  of  the  course  was  computer  intensive  and  the  students  had  opted  to  come  to  a classroom  lecture 
because  the  personal  interaction  was  necessary  for  their  understanding  of  the  material.  The  Michigan  study  found. 
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“They  [students]  primarily  reviewed  lectures,  although  a small  percentage  (9.1  percent)  used  the  online  lecture  as  a 
replacement  for  attending  class.”  (Brittain,  et  al.,  2006)  The  UH  study  found  a slightly  higher  percentage  (14%)  of 
students  were  using  the  podcasts  to  make  up  for  missed  lectures.  The  UH  study  determined  a number  of  reasons 
students  had  for  accessing  course  material  via  podcasts.  All  this  goes  to  illustrate  that  some  courses  will  suffer  a 
greater  drop  in  attendance  than  others,  but  data  indicates  that  podcasts  can  do  an  adequate  job  of  keeping  student 
grades  consistent.  (Figure  4) 

Not  everyone  will  be  satisfied  with  consistent  grades  as  a balance  to  lack  of  attendance,  nor  should  they  be. 
Students  miss  out  on  a great  deal  when  they  are  not  in  class.  From  an  educator’s  standpoint,  it  is  incredibly  difficult 
to  identify  muddy  areas  if  students  are  not  present  to  ask  questions.  As  Robert  Kadel  writes  in  an  article  in  Learning 
and  Leading  with  Technology,  “an  MP3’s  eyes  can  not  light  up  when  it  suddenly  grasps  a difficult  concept,  and  it 
cannot  furrow  its  brow  in  confusion.  Even  the  most  seasoned  professors  require  such  cues  in  determining  if 
information  needs  to  be  repeated.”  (Kadel,  2006)  Kadel  also  points  out  that  just  because  a student  is  in  class  does  not 
mean  they  are  paying  any  attention  to  the  lecture  at  hand.  So  the  question  arises,  “How  do  I keep  students  coming  to 
class  if  I podcast?”  One  answer  is  to  only  make  the  podcasts  available  the  week  before  exams.  Some  Universities 
hold  podcast  material  for  a month  before  distributing  it  to  students,  the  idea  being  that  students  will  come  to  class 
and  only  use  the  podcasts  for  study  review.  Another  method  is  to  offer  things  during  class  lectures  that  are  not 
included  on  distributed  podcasts.  This  can  and  often  does  create  a need  for  editing  if  an  instructor  is  recording 
classroom  lectures  for  podcasting.  The  added  time  editing  should  be  worth  it  if  students  attend  classes  know  that 
they  will  be  receiving  something  more  than  is  available  to  them  via  podcasts.  Another  option  is  to  use  podcasts  as 
purely  supplemental  information,  but  the  University  of  Houston  and  University  of  Michigan  studies  indicate  that 
students  do  benefit  from  being  able  to  review  entire  lectures  before  exams. 

Tips  for  Implementation 

Explain  the  Technology  - Be  sure  to  offer  an  explanation  of  “Podcasting”,  how  it  affects  the  course  workload,  and 
stress  the  potential  benefits  of  the  technology.  Remember,  podcasting  will  work  for  some,  it  will  not  work  for  others. 
Take  the  time  at  the  beginning  of  the  semester  to  get  students  interested  in  the  technology,  and  periodically  remind 
them  the  technology  is  out  there. 

Find  what  works  - Not  every  class  is  the  same.  Different  courses  require  different  solutions.  Instructor  created 
audio  only  podcasts  of  lectures  might  be  all  that  is  required  of  one  course  while  student  created  video  podcasts  might 
serve  another  course  more  effectively.  Think  about  what  is  best  for  the  course.  Ease  into  production  and  as  things 
become  more  comfortable  they  can  become  more  complicated  if  need  be. 

Keep  it  short.  - Time  is  a precious  commodity  in  the  world  today.  In  an  article  in  The  Chronicles  of  Higher 
Education,  Patrick  Jackson,  an  assistant  professor  at  American  University,  discusses  pre-recording  lectures  with  the 
expectation  students  will  listen  to  a lecture  before  attending  class  and  come  prepared  for  discussion.  “Think  of  how 
much  classroom  time  you  would  save  if  you  didn’t  have  to  lecture  anymore  . . . You  free  up  all  this  interactive 
personal  space  between  you  and  your  students.  It  changes  the  classroom  experience.”  (Campbell,  2005)  This  is  an 
interesting  implementation  of  the  technology,  but  one  that  has  numerous  potential  hazards.  If  there  is  a concern  that 
podcasting  lectures  will  keep  students  out  of  the  classroom,  an  equal  amount  of  concern  should  be  given  to  the  pre- 
class podcasts  replacing  reading.  Many  students  plan  their  semesters  with  an  idea  of  the  amount  of  time  each  class 
will  occupy  outside  of  lecture  periods.  If  a student  is  taking  15  hours  of  traditional  face  to  face  classes,  they  have  a 
certain  expectation  of  the  time  requirements  of  those  classes.  If  instructors  introduce  podcasts  to  be  listened  to 
before  each  class  as  required  material,  in  addition  to  course  readings,  the  result  is  a doubling  of  the  time  expected  of 
the  course.  The  consequences  could  be  frustration,  resentment  and  even  hostility  to  the  goals  of  the  professor.  The 
available  data  indicates  that  podcasts  should  be  between  20  - 25  minutes  or  even  shorter  if  possible.  From  a 
technological  standpoint,  longer  lectures  should  be  broken  into  multiple  files.  This  keeps  file  size  manageable, 
download  times  reasonable  and  students  can  engage  with  the  material  in  reasonable  increments  to  match  a mobile 
lifestyle  and  work  with  the  classroom  on  the  go.  The  pedagogical  benefits  of  delivering  entire  lectures  before 
classroom  discussion  is  an  area  worth  closer  examination  and  study.  There  are  only  so  many  hours  in  a day.  Society 
is  already  suffering  from  information  overload.  Just  because  podcasting  provides  the  ability  to  go  into  greater  depth 
than  was  possible  in  the  past  does  not  mean  that  students  will  have  the  time  to  digest  all  the  information  provided.  If 
a professor  wishes  to  go  into  a dissertation,  go  for  it,  but  make  it  a separate  file  and  indicate  that  it  is  there  for  those 
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who  want  the  extra  information.  The  students  that  are  truly  interested  and  will  benefit  the  most  from  the  information 
will  find  it. 

Lay  the  foundation  and  allow  it  to  be  revisited  - The  first  chapter/lecture  of  a semester  is  often  the  most  important 
and  lays  the  groundwork  for  the  entire  semester.  The  first  chapter/lecture  can  also  be  the  one  that  is  most  easily 
muddied  by  the  class  finding  itself  at  the  start  of  a term.  A good  recap  of  the  groundwork  for  the  course  can  help  all 
the  students  to  remain  focused. 

Breaking  traditional  educational  models  and  introducing  new  ways  of  learning  does  not  happen  over  night  - “Give 
them  [students]  the  option  of  reviewing  [course  material]  while  they  are  doing  laundry  or  waiting  for  a bus,  and  they 
might  just  take  you  up  on  it.”  (Read,  2005)  They  might  do  that,  but  the  data  suggests  that  the  majority  are  not.  The 
idea  of  learning  on  the  go  is  foreign  to  many  students.  Instructors  introducing  podcasting  need  to  take  the  time  to 
explain  the  benefits,  often  more  than  once. 

Servers  go  down  and  computers  crash,  so  have  a back-up  plan  ready  - A simple  glitch  in  the  network  can  throw  an 
entire  semester  out  of  whack.  Introduce  the  technology  well  before  it  is  needed  for  class.  Don’t  assume  everyone 
will  have  a working  knowledge  of  the  technology  or  an  ability  to  access  it. 

E-mail  when  new  podcasts  are  available  - The  information  is  useless  if  students  aren’t  getting  it.  E-mail  reminders 
when  new  podcasts  go  online  can  be  very  beneficial  to  students.  Podcasts  can  fall  victim  to  their  own  efficiency. 
The  subscription  feature  allows  for  new  podcasts  to  automatically  be  made  available  to  students,  but  if  a student  is 
unaware  that  the  new  material  is  there,  they  might  have  the  information  but  never  even  know  it. 

Conclusion 

The  data  suggests  podcasting  can  be  and  effective  pedagogical  tool.  Careful  implementation  should  allow  for  new 
and  diverse  uses  for  the  technology  in  and  out  of  the  classroom.  The  creation  of  on  demand  supplements  based  on 
careful  analysis  of  problem  areas  for  students  can  have  numerous  educational  benefits.  Students  can  review  material 
before  tests  or  after  lectures  to  gain  extra  insight  or  a better  understanding  of  classroom  subjects.  The  most  important 
factor  in  the  equation  is  learning  outcomes.  Will  podcasting  help  students  retain  information,  understand 
information  and  maybe  even  enjoy  the  educational  process  along  the  way?  The  numbers  say  yes. 
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Figure  1 - Students'  preferred  format  of  podcasts  when  using  portable  media  devices  to  access 
instructional  materials  while  involved  in  other  activities 
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Figure  2 - UH  Students'  response  to  the  statement  "I  found  the  addition  of  podcasts  to  be  helpful 

in  understand  course  content." 
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Figure  3 - UH  Students’  response  to  the  statement  "I  prefer  the  combination  of  iPod/MP3  player  course 
delivery  with  the  face-to-face  classroom  experience." 
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Figure  4 - Reasons  given  by  UH  students  for  accessing  podcast  material. 
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A Game-Based  Summer  Math  Camp:  Can  Learning  Be  Fun? 

Fengfeng  Ke 

University  of  New  Mexico 

Abstract:  This  paper  presents  findings  from  a case  study  which  experiments  with  the  use  of  educational  computer 
games  in  a summer  math  program  to  facilitate  4th-6th  graders’  math  skills  and  positive  attitudes  toward  math  learning. 
Using  observation,  interviewing,  and  think-aloud,  this  study  looks  at  learners’  lived  experiences  with  math  game 
playing  and  explores  how  game  play,  learning  task,  and  instructional  support  should  be  integrated  in  a game-based 
learning  system  to  support  an  engaging,  effective  learning  experience. 

Introduction  and  Theoretical  Background 

Computer  games  were  proposed  as  a potential  learning  tool  by  both  educational  researchers  (e.g.,  Barab, 
Thomas,  Dodge,  Carteaux,  & Tuzun,  2005;  Betz,  1996;  Gee,  2003;  Kafai,  1995;  Malone,  1981;  Rieber,  1996;  Squire, 
2003)  and  game  design  researchers  (e.g.,  Prensky,  2001;  Aldrich,  2005).  Frequently-cited  arguments  for  using 
computer  game  in  education  are:  (a)  computer  games  can  invoke  intense  engagement  in  learners;  (b)  computer 
games  can  encourage  active  learning  or  learning  by  doing;  and  (c)  computer  games  can  foster  collaboration  among 
learners. 

On  the  other  hand,  those  who  are  skeptical  toward  game-based  learning  contend  that  the  effectiveness  of 
computer  games  on  learning  is  still  a mystery.  Several  major  reviews  on  educational  games  (Dempsey,  Rasmussen, 

& Lucassen,  1996;  Ernes,  1997;  Harris,  2001;  Randel,  Morris,  Wetzel,  & Whitehill,  1992)  indicated  no  clear  causal 
relationship  between  academic  performance  and  the  use  of  computer  games.  A common  skepticism  on  using 
computer  games  for  learning  purpose  lies  in  the  lack  of  empirically-grounded  framework  for  integrating  computer 
game  into  classrooms.  As  Squire  (2003)  discovered,  bringing  a computer  game  into  classrooms  may  raise  as  many 
issues  as  it  solves.  First,  playing  games  doesn’t  appeal  to  every  student.  Then,  students  may  be  distracted  by  game 
playing  and  thus,  not  achieving  the  learning  goals  (Miller,  Lehman,  & Koedinger,  1999).  Furthermore,  students  may 
fail  to  extract  intended  knowledge  from  a complicated  gaming  environment,  hence  fail  to  learn  (Squire,  2003). 
Finally,  game  design  researchers  such  as  Smith  and  Mann  (2002)  are  worried  that  making  games  where  the 
objective  is  to  facilitate  student’s  learning  will  risk  sacrificing  the  game  part  along  the  way,  hence  the  very  argument 
for  using  games  for  learning,  that  they  are  fun,  vanishes  along  with  the  game  part.  Therefore,  the  key  question  still 
keeps  misty:  Will  computer  games  really  foster  an  engaging,  effective  learning  experience  in  classrooms? 

Limited  studies  were  conducted  to  answer  the  above  questions.  A recent  review  of  game -based  learning 
research  indicated  that  most  gaming  studies  focus  on  learning  conceptually,  such  as  general  reasoning,  creativity, 
and  decision  making,  which  does  not  demand  special  knowledge  of  subject  area  (Bateson,  1972).  In  other  words, 
many  current  games  used  for  facilitating  learning  lack  connection  to  curriculum  in  school.  The  content  in  these 
games  are  too  general  and  inappropriate  for  fulfilling  existing  curriculum  (Egenfeldt-Nielsen,  2003). 

Certain  researchers,  such  as  Barab  et  al.  (2005)  and  Squire  (2003),  did  start  to  examine  what  happens  with 
students  and  their  learning  processes  in  game-based  curricula  of  mathematics,  science,  and  history.  They  either 
applied  design-based  research  to  understand  and  improve  a game  design  for  instructional  use,  or  customized  a 
commercial  game  for  classroom  application.  A common  thing  they  shared  is:  the  games  used  are  microworlds  or 
simulation  games.  As  a complement  to  their  works,  the  present  study  focused  on  drill  and  practice  games.  Two 
reasons  underlie  the  selection  of  drill-and-practice  game  type:  (a)  computer  games  have  been  used  in  education 
primarily  as  tools  for  supporting  drill  and  practice,  yet  little  research  has  been  done  on  the  effectiveness  of  these 
games;  (b)  in  comparison  with  simulation  games,  drill  and  practice  games  are  easier  to  be  integrated  into  a 
traditional  curriculum  (Squire,  2003). 

Therefore,  this  study  intends  to  investigate  the  application  of  drill  and  practice  computer  games  in  a 
summer  school  math  program,  by  focusing  on  these  research  questions:  (1)  What  are  students’  lived  experiences  in 
game -based  learning  setting?  (2)  How  is  the  interaction  between  students,  game,  and  classroom  environment? 

Methods 

The  study  is  a phenomenological  examination  of  students  in  a game-based  learning  setting.  Using 
phenomenology,  the  study  is  intended  to  “construct  an  animating,  evocative  description”  of  participants’  game 
playing  experiences  and  perceptions  (Van  Manen,  1990,  p.  19).  Data  were  collected  in  multiple  forms  - in-field 
observation,  document  analysis,  and  think  aloud  verbal  protocol  - to  achieve  a data  triangulation.  The  researcher 
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subsequently  conducted  an  analysis  of  themes  in  order  to  explore  “the  deep  meaning  of  individual  subject’s 
experiences”  (Rossman  & Rallis,  1998,  p.  72).  Based  on  the  core  themes,  the  research  generated  working 
hypotheses  on  the  active  interaction  between  participants,  the  gaming  program,  and  the  external  learning  setting. 

Setting  Background 

ASTRA  EAGLE,  a series  of  web-based  games  developed  by  the  Center  for  Advanced  Technologies  of  the 
sampled  school  district,  was  used  in  this  study.  The  games  were  designed  to  be  drill-and-practice  programs  to 
reinforce  academic  standards  for  mathematics  required  by  “Pennsylvania  System  of  School  Assessment  (PSSA),” 
which  is  a standards-based  criterion-referenced  assessment  required  by  all  public  schools  in  the  Commonwealth  of 
Pennsylvania.  The  games  were  developed  using  Macromedia  Flash  and  will  run  in  any  recent  major  Web  browser. 
In  this  study,  eight  mathematics  games  within  the  ASTRA  EAGLE  set  that  target  4th-5th  students  were  used.  These 
mathematics  learning  games  contain  a variety  of  problems  to  be  solved,  such  as  measurement  problems,  comparing 
whole  numbers,  solving  simple  equations,  and  mapping  X and  Y coordinates.  In  some  games,  math  problems  are 
concealed  and  contextualized  in  stories  pertinent  to  school  students.  For  example,  in  a game  called  Cashier,  players 
need  to  play  as  a cashier  doing  math  calculation  of  money.  Differently,  in  other  games,  math  problems  are 
presented  in  independent  screens  as  puzzles  to  be  solved.  For  example,  a game  is  a Tic  Tac  Toe  board  game  where 
a game  player  plays  against  a opponent  and  wins  by  answering  math  questions  on  the  cards  correctly.  Math 
questions  are  multiple-choice  items.  Each  game  has  multiple  levels.  To  “conquer”  the  lower-level  unit  and  “bump 
up”  to  a higher-level  one,  students  need  to  answer  all  questions  of  that  level  correctly.  The  more  levels  one 
conquers,  the  higher  score  he/she  earns. 

Wilson  Elementary  in  the  sampled  school  district  is  an  academic  outperforming  school.  The  school  is 
located  in  the  rural  area  of  Pennsylvania,  with  K-6  grades,  47%  of  students  enjoying  free  or  reduced  lunch  (namely 
socio-economic  disadvantaged  in  this  paper),  and  92%  of  students  being  white.  In  the  summer  of  2005,  the  school 
held  a math  camp  for  students  of  4th  - 5th  grade.  Participation  was  voluntary.  The  camp  was  held  from  10am  to 
12pm  every  Tuesday  and  Thursday  for  five  weeks  during  June  and  July.  In  this  summer  math  program,  all 
participants  gathered  in  the  school  computer  lab,  each  interacting  with  an  Internet-connected  desktop  and  playing 
ASTRA  EAGLE  math  games  during  10  two-hour  sessions.  The  researcher  volunteered  to  be  the  coordinator  of  the 
program,  administering  and  managing  the  operation. 

Study  Participants 

Fifteen  4lh-5lh  grade  students  were  enrolled  in  the  summer  math  program  and  participated  in  this  research 
project.  They  were  10-13  years  old,  with  five  being  socio-economic  disadvantaged,  10  being  girls,  and  all  being 
white.  Participants’  pre-program  school  grades  were  collected.  Their  math  abilities  were  classified  into  four  levels 
- advanced,  proficient,  basic,  and  below  basic.  Out  of  the  15  students,  four  were  advanced,  six  were  proficient,  and 
five  were  basic  or  below  basic  in  math  achievement.  Participants  were  quested  on  their  prior  gaming  experience  and 
if  necessary,  trained  to  know  basic  computer  skills,  such  as  using  a mouse  to  click  buttons  on  the  computer  screen. 
At  the  beginning  of  the  summer  math  program,  all  participants  took  one  orientation  session  where  they  got  familiar 
with  the  gaming  environment  and  were  trained  to  do  think-aloud. 

Data  Collection  and  Analysis 

In-field  observation,  participants’  game -playing  records  analysis,  and  think-aloud  verbal  protocol  were 
employed  in  the  study  to  achieve  a triangulation  of  data.  Direct  observation  of  the  participants  went  through  every 
game -playing  session.  Concurrently,  participants  were  asked  to  do  think-aloud  that  intends  to  explore  their 
emotional  situation  and  cognitive  processes  when  interacting  with  game  features,  their  strategies  in  handling  math 
problems  in  the  games,  and  the  intentional  or  incidental  knowledge  they  constructed.  Participants’  game -playing 
logs,  indicating  their  on-task  time  and  gaming  scores,  were  archived  and  analyzed  every  week. 

Observation:  The  researcher  closely  observed  participants’  behaviors,  oral  and  body  language,  and  facial 
expressions  when  they  interacted  with  the  computer  program,  peers,  and  the  external  environment.  A semi- 
structured  observation  protocol  was  developed  to  guide  the  attention  during  observation,  though  the  actual 
observation  was  open  to  any  situational  changes. 

Think  aloud:  Based  on  Ericsson  and  Simon’s  (1993)  talk-aloud  method,  the  researcher  developed  a very 
open-ended  protocol  to  prompt  participants  to  report  what  came  to  their  mind  when  they  playing  the  computer 
games.  Participants  were  asked  to  say  whatever  they  are  looking  at,  thinking,  doing,  and  feeling,  as  they  went  about 
their  task.  Promoting  questions  include:  What  are  you  thinking  now?  Why  did  you  do  that?  How  are  you  feeling? 
Participants’  verbal  protocols  were  recorded  by  a mini-sized  digital  recorder.  Efforts  were  taken  to  enable 
participants  to  generate  a self-report  of  on-going  actions  without  being  interrupted  or  biased. 
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Document  analysis : the  computer  game  program  recorded  participants’  on-task  time,  the  numbers  of  math 
questions  they  had  tried  to  solve  and  questions  they  had  solved  correctly,  and  the  gaming  scores  they  earned.  These 
records  were  collected  every  week  and  coded. 

Data  analysis:  By  following  Van  Manen’s  (1990)  proposition  on  phenomenological  research,  the 
researcher  did  thematic  analysis  to  determine  salient  themes  that  stand  for  the  essence  or  structures  of  experience. 
The  researcher  employed  constant  comparison  of  participants’  responses  and  activities  with  the  goal  of  finding  the 
recurring  themes  and  organizing  the  data  into  systematic  categories  of  analysis.  The  statements  or  meaning  units 
that  emerged  as  possible  commonalities  from  the  data  were  forwarded  as  initial  themes  (Creswell,  1998)  and  coded 
using  Nvivo  software.  The  researcher  then  refined  these  themes  by  removing  overlapping  ones,  capturing  the  main 
thrust  of  each  theme’s  meaning,  and  re-examining  them  through  member  checking  (Guba  & Lincoln,  1994)  . 
Through  this  data  coding  process,  general  themes  emerged  and  were  synthesized  based  on  the  context. 

Findings  and  Discussion 

Three  general  patterns  on  participants’  experiences  in  game -based  learning  environment  emerged  through 
constant  comparison  thematic  analysis  process.  These  three  patterns  included  multiple  salient  themes  that  depicted 
how  participants,  cognitively  and  affectively,  interact  with  the  computer  educational  game,  peers,  and  the  external 
environment. 

Learning  with  Fun  or  Learning  versus  Fun? 

When  asked  about  their  feelings  toward  certain  computer  game,  participants  usually  stated,  “It’s  fun”  or  “I 
feel  bored,  it  needs  too  much  calculation.”  They  rarely  commented  about  the  learning  values  of  the  games,  unless 
they  were  prompted.  At  the  opening  day  of  the  math  camp,  they  showed  a lot  of  thrill,  “so  we  will  just  play  game? 
Cool. . .”  But  with  time  passing  by,  they  realized  the  games  were  actually  for  learning  purpose.  It  was  at  that  time 
that  quite  a few  participants  reported  being  disappointed  and  bored: 

“Oh. . . they  are  learning  games.”  (multiple  participants) 

“Can  we  play  some  other  games?”  (Amy,  5th-grade,  proficient  in  terms  of  math  competency) 

“What  kind  of  games  do  you  want  to  play?”  (Researcher) 

“Well,  games  that  are  fun.”  (Amy) 

“Don’t  you  think  this  game  is  fun?”  (Researcher) 

“Kind  of.  But  I don’t  like  the  questions  in  it.  1 had  to  think  hard  (about  the  questions).”  (Amy) 

“If  I passed  the  first  level  of  this  game,  can  I play  other  games  on  the  Internet?”  (Tom,  4th  grade,  basic  in 
terms  of  math  competency) 

In  the  participants’  minds,  the  goodness  of  a game  would  be  spoiled  by  its  learning  element.  This 
observation  confirms  a 2001  survey  (ESA,  Kirriemuir  & McFarlane,  2004)  on  the  main  reasons  for  gameplay, 
namely:  “87%  of  most  frequent  computer  and  video  game  players  said  the  number  one  reason  they  play  games  is 
because  its  fun”  (p.  5).  It  also  explains  why  certain  educational  gaming  researchers  recommend  “learning  by 
stealth”  - learning  can  only  be  enjoyable  when  it  is  concealed  within  games  and  thus  unconscious  to  the  learners 
(Prensky,  2001).  In  this  inquiry,  it  was  found  that  most  of  participants  contradicted  gaming  with  learning,  deeming 
the  former  as  play  whereas  the  latter  as  work  (Rieber,  1996).  Therefore,  when  facing  certain  drill-and-practice  math 
games  where  learning  part  is  clearly  segmented  from  game  dynamics,  the  participants  reacted  by  deeming  learning 
component  as  a foe  and  chose  to  simply  bypass  it.  As  a result,  wondering  mouse  - clicking  the  screen 
inconsiderately  to  move  around  the  problems  - was  their  frequently-observed  behavior. 

Wondering  mouse 

The  document  analysis  of  participants’  gaming  record  indicated  that  for  most  of  game  players,  the  ratio  of 
questions  solved  to  questions  tried  was  very  low  - one  out  of  ten  averagely,  and  the  time  taken  to  finish  a single 
question  was  unreasonably  short  - usually  in  seconds.  Such  a pattern,  as  confirmed  by  the  in-field  observation,  was 
due  to  that  participants  simply  did  wild  guessing  with  math  problems.  The  user  response  design  in  the  ASTRA 
EAGLE  games  - using  multiple-choice  item  - aided  this  wondering  mouse  behavior.  When  asked  why  and  how 
they  did  guessing,  the  participants  gave  the  following  reasons: 

1.  A mismatching  between  challenge  and  ability: 

■ “It’s  too  difficult.  I even  don’t  understand  the  problem.”  (Jack,  5th  grade,  below  basic  in  terms  of  math 
competency). 
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■ “(Recited  the  question)  what  is  the  next  number  in  the  following  sequence?  What  is  the  next  number  in 
the  following  sequence?  101,  94,  80,  52. ..mm. ..101  minus  94,  that  will  be  7. ..94  minus  7,  that  will  be 
86?  Or  no,  that  will  be  87. . .87.. mm. . .oh,  it’s  not  7,  so  101  minus  94  is  17?  Wait  a minute... it  is  7.  94 
minus  80,  is  14. . .oh. . .1  am  confused.  It  is  really  confusing.  All  right,  I will  just  guess.  Is  it  B?  ” (Ray, 
5th  grade,  proficient  in  terms  of  math  competency) 

This  observation  is  in  alignment  with  Malone’s  (1981)  proposition  on  optimized  challenge:  the  game’s 
difficulty  level  should  be  appropriate  with  respect  to  players’  levels;  otherwise  the  game  won’t  be 
intrinsically  motivating. 

2.  Guessing  is  fun,  just  like  gambling 

“I  like  guessing.  Guessing  is  fun  and  I am  good  at  it."  (Mary,  4th  grade,  basic  in  terms  of  math  competency) 
“You  should  think  about  this.”  (Researcher) 

“But  it’s  fun.  Calculation  is  boring.” 

3.  Avoid  cognitive-demanding  task 

■ “I  am  not  wildly  guessing.  I know  it  should  be  an  answer  smaller  than  9.  So  it  should  be  either  C or 
D.”  (Amy). 

■ “(Interpreted  the  question)  so  it  is  9 feet  3 inches  minus  6 feet  7 inches.  9 minus  6 is  3.  And  3 minus  7, 
3 minus  7?  Well,  the  answer  should  be  3 feet  and  inches.  It  should  be  A or  B. . .1  will  try  B. . .yeh. . .1 
got  it.”  (David,  4th  grade,  proficient  in  terms  of  math  competency). 

■ “(Recited  the  question)  what  is  the  next  number  in  the  following  sequence,  1,  7,  31,  127?  Huh,  this 
one  is  easy.  It  is  51 1,  C.”  (Jeff,  5th  grade,  advanced  in  terms  of  math  competency). 

“How  did  you  know?”  (Researcher) 

“The  ending  number.  It’s  1,  7,  1,  7,  so  the  next  number  will  be  ended  in  1,  too.  Huh,  I got  it  right,  and 
I don’t  need  to  do  calculation.” 

“Are  you  sure  this  trick  will  always  work?”  (Researcher) 

“At  least  it  works  now!” 

Evidently,  participants  tried  to  avoid  steps  that  need  effortful  calculation  and  thinking.  They  picked  up 
easy  steps  then  left  more  complicated  ones  to  luck.  Through  the  game  playing  process,  they  developed 
some  tricks  to  beat  the  game.  When  asked  how  they  were  doing,  they  said,  “We  are  better  answering  the 
questions.  We  know  the  tricks.”  As  observed,  cued  guessing  helped  game  players  earn  gaming  scores 
more  easily,  but  didn’t  help  them  learn. 

Playing  without  reflection 

It  was  found  that  when  interacting  with  ASTRA  EAGLE  drill-and-practice  games,  very  few  participants 
did  reflection  of  their  performance  to  gain  lessons  for  future  problem  solving.  They  would  attend  to  feelings  - 
expressing  happiness  or  disappointment  with  a failure  or  success  in  problem  solving,  then  instantly  move  on.  Both 
observation  and  think-aloud  protocol  indicated  that  most  of  participants  lacked  a reflection  process  to  conduct 
performance  analysis,  new  knowledge  generation,  evaluation,  and  integration,  which  are  essential  for  trial-and-error 
learning  - the  major  knowledge-construction  format  for  game-based  learning  (Gee,  2003).  Such  a happening  was 
mainly  due  to  two  reasons: 

1 . The  games  reward  the  player  based  on  the  total  number  of  questions  answered  correctly,  rather  than  a 
ratio  of  questions  answered  to  questions  tried.  Therefore,  to  earn  a higher  gaming  score,  participants 
simply  handled  questions  as  quickly  as  possible: 

“(Spent  one  second  viewing  the  question)  oh  my  god,  the  problem  is  really  long,  too  wordy.  All 
right. . .(wondered  the  mouse  and  clicked  one  choice). . .oh,  I win,  yeh. . .(clicked  the  Continue  button).” 
“Oh,  I got  it  wrong.  Whatever. . .let’s  see  this  one  (started  another  question).”  (Sam,  4th  grade,  below 
basic  in  math  competency) 

It  can  be  interpreted  that  the  desire  to  win  has  twisted  the  participant’s  normal  learning  speed,  pushed  his 
pacing  with  the  task,  and  thus  cutting  off  the  valuable  reflective  thinking  time. 

2.  The  feedback  in  the  games  is  summative  rather  than  informative.  When  one  solved  a problem 
correctly,  the  screen  would  come  up  with  a congratulation  message  with  hands-clapping  sound  effect; 
when  one  failed  a problem  solving,  the  screen  would  present  a “you  are  wrong”  message  and  indicate 
the  right  answer.  No  further  information,  such  as  adapted  informative  feedback  for  player’s  trial-and- 
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error  learning,  was  presented.  As  a result,  participants  didn’t  get  enough  information  to  retrospect  and 
chew  on.  In  most  times,  they  could  only  say,  “Oh,  what  the  hell. . . Why  it’s  C?  I don’t  understand.” 


When  will  one  learn  with  fun? 

In  certain  cases,  tasks  of  learning  and  getting  fun  did  team  up.  For  instance,  in  a game  called  Treasure 
Hunt,  participants  needed  to  plot  coordinates  on  an  XY  graph  to  locate  the  treasure  spot.  Participants’  think-aloud 
protocol  did  indicate  clear  cognitive  thinking  when  they  searched  for  the  spot: 

X9  Y7,  X9,  Y7  (repeating  the  question). . . X is  here. . .XI,  X2,  X3..X9;  Y is  going  here,  Y6,  Y7.  Yah, 
there. . .now  I had  to  figure  out  how  to  get  around  the  tree  to  go  to  Y7. . .got  it  (he  was  able  to  use  the  arrow 
key  to  move  the  shuffle  around  the  tree  and  get  it  to  the  right  spot)  . . . Y7(he  was  checking  the  Y 
coordinate). . .X9(he  was  checking  the  X coordinate). . . Yeh,  I found  the  treasure!  (Sam) 

As  this  example  indicated,  the  participant  had  employed  cognitive  thinking  steps  of  question  interpretation, 
principles  execution,  and  self-monitoring  with  checking. 

It  was  also  observed  that  game  players  would  be  more  willing  to  employ  effortful  cognitive  thinking  when 
they  considered  the  problem  in  hand  as  manageable  (appropriate  in  difficulty  level),  or  high-stake  (solving  the 
problem  or  not  would  decide  whether  they  would  win  a game  unit  or  not).  For  instance,  Mary,  the  participant  who 
enjoyed  doing  wild  guessing  most  of  the  time,  was  able  to  “think  about”  certain  questions  at  certain  game  unit 
because  she  “knew  everything”  about  those  questions.  In  addition,  it  was  found  that  most  game  players  were  most 
careful  in  answering  the  last  question  of  every  game  unit: 

Great,  with  one  more  question  I will  be  able  to  beat  Fuzzy  (the  fictitious  opponent  in  the  game). . .(read  the 
question  slowly). . . “fall  down  by”. . .so  it  is  a factor. . .minus,  or  no,  it  should  be  multiply. ..  15  times  5. . .it’s 
60. . .Um,  it’s  not  there  (in  the  multiple  choices),  so  I must  be  wrong. . .all  these  numbers  are  small. . .so,  it  is 
divide...  15  divides  5...  it’s  3...  B!  Yeh!  I win,  I win!  (Mark,  5th  grade,  advanced  in  terms  of  math 
competency) 

As  this  verbal  protocol  depicted,  in  certain  situation  participants  were  engaged  in  effortful  cognitive 
thinking,  especially  when  they  considered  the  question  in  hand  as  a high  stake  for  their  sense  of  winning. 

Play-Based  Communication 

It  was  observed  that  collective  game-playing  facilitated  peer  communication.  During  every  two-hour 
session,  game  players  were  very  active  in  exchanging  game  scores,  expressing  feelings  about  the  games,  and  in 
certain  cases,  doing  social  talk  that  was  irrelevant  to  the  learning  tasks.  It  was  noted  that  these  communications  and 
peer  activities  were  mostly  play-based  rather  than  learning-oriented. 

Peer  scaffolding  is  difficult 

Both  observation  and  think-aloud  protocols  indicated  that  elaborating  mental  processes  in  mathematical 
problem  solving  was  an  effortful  task  for  young  participants.  Without  sustained  press  and  prompting  for  meaning 
and  explanation,  one  could  not  enable  a 4th-  or  5th-  grader  to  give  mathematical  thinking  elaboration: 

“It’s  B”  (David) 

“Why?  How  did  you  get  this?”  (Researcher) 

“I  did  subtract.  Subtract  5 feet  7 inches  from  10  feet  2 inches.” 

“Why  did  you  do  subtract  instead  of  add,  or  other  operations?” 

“Because. . .because  you  want  to  make  the  number  small. . .” 

“Why?  Is  there  any  key  word  in  the  sentence  telling  you  that  you  should  use  subtract?” 

“In  the  sentence,  it  said  ‘cut’”. 

“I  see.  Then  how  did  you  subtract  5 feet  7 inches  from  10  feet  2 inches?” 

“10  minus  5. . . and  then  2 minus  7. . .wait,  because  2 is  too  small,  so  you  have  to  borrow  1 feet,  so  it’s  12 
plus  2,  14;  then  14  minus  7 is  7,  and  then  9 minus  5,  is  4 feet.  So  4 feet  7 inches.” 

“Why  it  is  12  plus  2?” 

“Because...” 

As  this  conversation  indicated,  it  took  the  participant  quite  some  time  and  efforts  to  elaborate  every 
cognitive  step  contained  in  math  problem  solving.  Not  every  potential  student  tutor  was  able  to  do  that;  not  every 
potential  tutee  was  able  to  prompt  for  such  an  explanation.  In  consequence,  peer  scaffolding  was  not  an  easy 
happening.  More  often,  a participant  would  throw  his/her  peers  an  answer  directly,  rather  than  explaining  his/her 
mental  model  or  problem  solving  process:  “Just  trust  me.  Choose  A.”  (Mark). 
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Boys  versus  girls 

Interestingly,  the  researcher  found  that  during  game  playing,  boys  tended  to  report  information  related  to 
the  games,  such  as  game  scores  earned,  level  of  game  units  conquered,  obstacles  met,  and  tricks  to  handle  the 
problems.  Differently,  girls  spent  more  time  exchanging  feelings  and  attending  to  social  communication  that  was 
irrelevant  to  the  games.  Sometimes  the  researcher  had  to  interrupt  those  game-irrelevant  talks  and  remind  them  to 
focus  on  the  games.  Such  an  observation  confirmed  the  finding  of  an  earlier  gaming  study  (Greenfield,  1984)  that 
computer  game  is  a facilitator  for  social  communication,  and  complemented  Tobin’s  (1998)  argument  that  boys’ 
gaming  was  not  simply  a process  of  ‘playing  the  game’  but  embedded  in  social  interactions. 

Quiet  achiever 

Among  the  group  of  game  players,  there  were  several  quiet  achievers.  Brian  was  one  of  them.  He  always 
chose  to  sit  in  one  corner  and  played  the  games  by  himself.  He  rarely  talked  to  others  during  the  game -playing 
process;  neither  would  he  proudly  report  his  progress  to  the  researcher,  like  some  other  kids  would  do.  But  his 
game -playing  records  indicated  that  he  had  a surprisingly  high  attainment.  He  was  advanced  in  terms  of  math 
competency.  When  prompted,  he  could  give  a clear  explanation  of  his  mathematical  thinking  process,  yet  he  rarely 
offered  to  help  others.  The  researcher  had  tried  to  encourage  him  to  join  in  peer  collaboration,  and  arranged  for  him 
to  sit  beside  the  peers  who  could  use  his  help.  But  it  didn’t  work  as  expected:  he  still  chose  solitary  game  playing 
and  would  go  back  to  his  own  corner  next  time.  Maybe  for  such  a quiet  achiever,  solitary  playing  is  the  most 
comfortable  way  to  game -based  learning. 

Offline  Learning  Tools 

Another  interesting  finding  from  in-field  observation  was  that  participants  used  offline  tools  to  assist 
computer-game-based  online  problem  solving.  Generally  there  were  two  offline  tool  options:  pencil  with  paper,  and 
calculator.  As  observed,  participants  used  calculator  most  of  the  time.  A big  reason  was  the  limitation  of  work 
space:  game  playing  took  place  in  the  school  computer  center  where  desk  space  was  occupied  by  computers  and 
keyboards,  leaving  no  room  for  paper  and  pencil  usage.  Due  to  the  preclusion  of  paper  and  pencil,  participants  at 
game  playing  missed  a necessary  tool  that  would  assist  math  problem  solving  in  terms  of  mapping  multiple 
componential  steps  required  by  a complicated  word  problem.  In  spite  of  the  researcher’s  encouragement  (“You  can 
use  paper  and  pencil”),  quite  a few  participants  used  calculator  only.  The  exceptions  were  the  participants  with 
higher  prior  knowledge  (advanced  or  proficient  in  math  competency),  who  actually  used  paper  with  pencil  more 
than  calculator.  Therefore,  a question  germane  to  game-based  learning  is  how  the  management  of  physical,  external 
classroom  environment  may  assist  effective  learning  activities,  either  online  or  offline. 

Conclusion 

Conclusively,  this  qualitative  inquiry  indicated  that  game-based  learning  is  a systematic  process  that 
operates  through  the  interaction  between  a game  player,  the  computer  game  program,  his/her  peers,  and  external 
physical  environment.  Some  particular  educational  game  design  and  application  issues,  as  this  study  reveals,  should 
be  taken  care  of  in  order  to  enable  students  to  achieve  an  engaging  and  effective  learning  experience  in  a game- 
based  classroom: 

1.  The  connection  between  the  challenges  in  the  game  with  students’  competency  level  plays  an 
important  role  in  engaging  students  in  effortful  thought  processes. 

2.  The  design  of  learning  tasks  embedded  within  a computer  game  should  be  an  integral  part  of  the  game 
dynamics  design,  thus  making  learning  and  fun  incorporated  rather  than  conflicted. 

3.  User  response  should  be  designed  to  hinder  wild  guessing  behavior.  For  example,  multiple-choice 
items  should  be  avoided. 

4.  Certain  reflection-support  features,  such  as  adapted  informative  feedback,  reflection-prompting 
assignment,  and  explicit  rewards  for  reflective  thinking,  should  be  embedded  in  a computer 
educational  game  to  encourage  learning  reflection. 

5.  Students  should  get  relative  training  on  cognitive  elaboration  and  peer  scaffolding. 

6.  Classroom  or  the  learning  center  should  be  arranged  to  support  game-based  online  and  offline  learning 
activities. 
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Online  Teaching  Strategies  Lived  and  Told:  A Narrative 
Phenomenological  Approach 


Hee- Young  Kim 
SUNY  Cortland 

Abstract: 

Granting  online  instructors’  voices  can  serve  as  main  sources  of  research  data,  this 
interpretive  phenomenological  study,  based  on  narrative  inquiry,  discusses  how 
instructors  try  to  provide  a sense  of  “online  instructional  immediacy”  in  the  process, 
course  design,  course  presentation,  use  of  technology,  and  teaching  interaction  in  totally 
different  social,  cultural,  and  pedagogical  modes  from  those  of  the  conventional 
classroom.  The  finding  of  this  study  details  the  online  teaching  strategies  with  which 
instructors  strived  to  compensate  the  removal  of  verbal  and  non-verbal  cues  in  their 
instruction  to  promote  more  motivated,  interactive,  effective  learning  environments.  The 
following  three  overarching  themes  are  identified  and  are  used  to  describe  online 
instructional  immediacy  strategies  found  in  online  instructors’  practice:  1)  affective 
interaction,  2)  cognitive  interaction,  3)  technology  adoption,  and  4)  course  presentation 
and  organization.  The  narrative  findings  hold  promise  for  better  online  instruction 
humanized,  personalized,  caring  with  their  shared  empirical  approaches.  This  is  the  first 
study  conducted  with  narrative  inquiry  method  in  Web-based  contexts. 

Introduction 

We  as  online  educators  are  seeking  to  develop  a new  paradigm  of  online  pedagogy,  fitting  together 
the  pieces  of  a puzzle  that  is  yet  unknown.  Online  instructors’  ways  of  teaching,  like  those  of  many 
traditional  teachers,  are  the  collections  of  habits  determined  by  trial  and  error  concerning  such  topics  as 
time  effectiveness,  students’  responses,  and  availability  in  systems.  Instructors  affirm  or  reject  a segment  of 
their  pedagogy  depending  on  whether  it  works  in  practice. 

Although  there  has  been  considerable  emphasis  in  online  distance  education  on  foregrounding  the 
online  instructors’  roles  and  online  teaching  guidelines,  little  attention  has  been  paid  to  knowledge 
generated  by  online  teaching  practitioners.  What  is  missing  from  the  knowledge  base  for  online  teaching 
are  the  voices  of  the  online  instructors  themselves.  Granting  online  instructors’  voices  to  serve  as  main 
sources  of  research  data,  this  study  aimed  at  exploring  how  online  instructors’  personal  practical  knowledge 
(Clandinin,  1986)  is  shaped  by  their  experiences  in  online  teaching.  With  particular  interest  in  social 
presence  in  online  instruction,  this  article  discusses  how  instructors  try  to  provide  a sense  of  “online 
instructional  immediacy”  in  the  process,  course  design,  course  presentation,  use  of  technology,  and 
interaction  in  the  context  of  social,  cultural,  and  pedagogical  modes  totally  different  from  those  of  the 
conventional  classroom  due  to  the  lack  of  physical  presence  of  both  teacher  and  student.  I borrowed  the 
construct  of  “teacher  immediacy”  from  the  research  in  conventional  classroom  contexts  and 
reconceptualized  this  notion  as  “online  instructional  immediacy.”  In  this  study,  “online  instructor 
immediacy”  is  defined  as  pedagogical,  administrative  actions  an  instructor  takes  throughout  an  online 
course  to  increase  the  students’  sense  of  human  interaction,  instructor  presence,  caring  and  connectedness, 
toward  the  end  of  successful  online  teaching. 

A narrative  account  of  online  instructors  ’ personal  practical  knowledge  might  help  to  overcome 
barriers  which  make  online  education  fail.  The  ultimate  aim  of  this  study  was  to  characterize  online 
instructors’  practical  knowledge  as  a set  of  rules,  principles,  directions  and  maxims  that  could,  in  turn, 
inform  novices’  and  even  experienced  instructors’  online  teaching  (Oakeshott,  1962)  in  relation  to 
instructional  immediacy. 
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Background  of  the  Study 


During  my  dissertation  journey  I took  a turn  from  quantitative  research  method  to  qualitative 
method.  1 had  embarked  on  my  quantitative  research  with  the  research  question  of  how  to  compensate  for 
the  lack  of  face-to-face  interaction  from  which  most  online  instruction  suffers.  Initially  the  research  design 
was  an  experimental  research  study  to  compare  the  perceptions  and  effectiveness  of  teacher  immediacy 
behaviors  for  distance-education  undergraduate  and  graduate  students  who  received  enhanced  teacher 
immediacy  online  instruction  versus  those  who  received  typical  online  instruction.  The  treatment  was 
training  of  online  instructors.  In  the  process  of  organizing  the  online  training  materials,  I discovered  the 
shortcomings  of  the  training  items  when  interviewing  one  online  practitioner.  What  the  practitioner  told  me 
showed  a discontinuity  between  what  is  recommended  in  research  literature  and  what  is  executed  in  online 
classrooms.  It  also  showed  online  instructors’  ambivalence  about  the  claims  of  academic  research  and  a 
general  lack  of  information  about  online  classroom  life  (Cochran-Smith  & Lytle,  S,  1990).  Many 
prescribed  guidelines  presented  in  the  body  of  literature  would  not  be  feasible  for  online  teaching 
practitioners.  My  research  shifted  gears,  therefore,  to  search  for  teachers’  personal  reflective  knowledge 
shaped  and  reshaped  through  experiences. 

Theoretical  Framework 

Online  teaching  and  learning  is  called  e-learning  because  knowledge  is  electronically  delivered. 
However,  a myriad  of  evidence  shows  that  online  interaction  needs  the  human  touch  to  be  effective  (Boser, 
2004;  Palloff  & Pratt,  1999;  Holmberg,  1989).  I focused  on  how  online  instructors  conceptualize  social 
aspects  of  online  teaching  by  looking  through  the  lens  of  “instructional  immediacy.”  Two  issues  appear 
most  relevant  and  serve  as  main  pillars  of  this  study.  One  is  the  importance  of  experience,  story,  and 
narrative  inquiry.  It  is  based  on  the  assumption  that  experienced  teachers  possess  a certain  kind  of 
knowledge,  knowledge  that  is  practical,  experiential,  and  shaped  by  a teacher’s  purposes  and  values  (Elbaz, 
1983;  Clandinin,  1986).  Another  pillar  of  this  study  was  the  notion  that  instructional  immediacy,  initially 
proposed  by  Mehrabian  (1967,  1971),  Andersen  (1979),  Gorham  (1988),  Richmond,  Gorham  and 
McCrosky  (1987)  and  defined  as  a teacher’s  psychological  and  physical  closeness,  affects  students’ 
learning,  satisfaction  and  instructional  effectiveness. 

1.  The  First  Pillar  for  Theoretical  Background: 

Experience,  Personal  Practical  Knowledge,  and  Narrative  Inquiry 

Based  on  Dewey’s  (1938)  idea  of  education,  the  main  focus  of  this  study  is  to  reconstruct  online 
instructors’  experiences  to  “reform  knowledge”  in  theory  and  practice.  Reconstruction  of  experiences 
requires  meaning  making.  The  meaning  making  takes  place  while  I thread  the  stories  of  the  lived 
experiences  of  online  instructors,  and  while  the  online  instructor  participants  distill  their  experiences  via  a 
process  of  reflection.  Online  instructors  transform  their  experiences  from  mere  circumstances  to  lived 
experiences  with  their  capacity  to  bestow  meaning,  to  be  reflective,  and  to  take  action  (Britzman,  2003). 
This  study  seizes  reflections  on  experiences  which  give  rise  to  valuable  knowledge  for  both  theory  and 
practice. 

The  term  “personal  practical  knowledge,”  defined  and  developed  by  Clandinin  and  Connelly 
(1988),  will  be  used  to  siginify  the  empirical  knowledge  of  online  instructors  generated  by  their  own 
experiences  and  reflections.  The  term  is  based  on  the  assumption  that  online  instructors  hold  critical 
knowledge  to  inform  the  world  of  research  and  practice.  Personal  practical  knowledge  is  found  in  the 
online  instructors’  practice. 

In  this  study,  following  Bruner  and  Eisner,  I use  narrative  as  a means  of  conducting  research,  and 
as  a vehicle  to  “organize  online  instructors’  understanding  and  memories  of  events”  in  their  practice 
(Bruner,  1986;  Polkinghorne,  1988).  The  rationale  for  the  use  of  narrative  is  based  on  the  belief  that 
narrative  has  extended  the  capacity  to  encompass  more  experiences  and  questions  (Bruner,  1986).  Although 
experience  cannot  be  recovered,  it  can  be  narrated.  (Brodkey,1987).  For  the  reasons  mentioned  above,  I 
adopt  Clandinin  and  Connelly’s  (2000)  narrative  inquiry  method  for  exploring  online  instructors’ 
transformative  experience.  Narrative  inquiry  is  a way  of  understanding  experience.  This  study  is  not  about 
proving  hypotheses,  inventing  new  ideas,  or  writing  new  theories.  It  is  a process  of  reconstructing 
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experiences  (Dewey,  1938),  legitimating  our  professional  memory,  and  returning  it  to  experiences  as  a 
resource  for  the  education  of  professionals  (Schon,  1983). 

2.  The  Second  Pillar  of  the  Theoretical  Background:  Teacher  Immediacy 

To  explore  the  possibility  of  human  connection  (Palloff  & Pratt,  1999),  and  of  emotional 
involvement  and  feelings  of  personal  relation  between  the  teaching  and  learning  parties  (Holmberg,  1989) 
in  online  education  environments,  the  construct  of  “instructional  immediacy”  is  used  in  the  study.  There  is 
a growing  body  of  evidence  that  instructional  immediacy  in  traditional  learning  contexts  plays  a central  role 
in  how  teachers  increase  positive  learning  outcomes  including  student  affective  learning  (Kearny,  Plax,  & 
Wendt-Wasco,  1985),  instructional  effectiveness  (Mehrabian,  1967,  1971;  Andersen,  1979),  and  cognitive 
learning  (Gorham, 1988).  Furthermore,  instructional  immediacy  has  been  found  to  contribute  to  students’ 
satisfaction  and  learning  in  an  interactive  television  class  (Hackman  and  Walker,  1990). 

The  Conceptual  Definition  of  Immediacy 

The  immediacy  construct  has  been  developed  by  (Mehrabian,  1969)  to  refer  to  those  communication 
behaviors  that  “enhance  closeness  to  and  nonverbal  interaction  with  another.”  Immediacy  behaviors  are 
behaviors  that  reduce  the  distance  between  people.  The  distance  reduction  can  be  accomplished  by 
decreasing  the  actual  physical  proximity  or  by  reducing  psychological  distance  (Mehrabian,  1971a). 
Mehrabian  (1971a,  p.l)  bases  this  conceptual  framework  on  approach  and  avoidance  principles.  Simply 
stated,  “people  are  drawn  toward  persons  and  things  they  like,  evaluate  highly,  and  prefer,  and  they  avoid 
or  move  away  from  things  they  dislike,  evaluate  negatively,  or  do  not  prefer.”  The  immediacy  can  be 
enhanced  by  increasing  mutual  sensory  stimulation  (visual,  tactile,  auditory,  and  olfactory  information)  and 
reduce  distance  by  increasing  the  number  of  available  and  utilized  communication  channels,  since 
“Communication  channels  are  the  means  by  which  one  conveys  his  thoughts  and  feelings  to  another” 
(Mehrabian,  197 la,  p.70) 

A person  can  convey  immediacy  or  non-immediacy  nonverbally  (physical  proximity,  formality  of 
dress,  and  facial  expression)  as  well  as  verbally.  Nonverbal  immediacy  behaviors  cues  include:  eye  contact, 
gesture,  relaxed  body  position,  directing  body  position  toward  students,  smiling,  vocal  expressiveness, 
movement,  and  proximity  (Andersen,  1979).  Particularly  significant  verbal  immediacy  cues  were:  use  of 
humor;  praise  of  student  work,  actions  or  comments;  frequency  of  initiating  and/  or  willingness  to  become 
engaged  in  conversation  before,  after,  or  outside  of  class. 

Instructional  Immediacy  in  Online  Education 

Despite  the  body  of  evidence  examining  instructional  immediacy  for  producing  positive  student 
affect  and  subsequent  teacher  effectiveness  in  conventional  classrooms,  it  is  important  to  understand  that 
we  can  relate  instructional  immediacy  to  positive  effect  on  learning  and  teaching  in  online  courses,  since 
Web-based  learning  creates  a learning  climate  which  is  totally  different  from  that  of  the  conventional 
classroom.  Recent  work  (Melrose  & Bergeron,  2006;  Baker  & Woods,  2004;  Arbaugh,  2001; 

Gunawardena,  1995)  highlights  instructional  immediacy  as  a valuable  means  associated  with  student 
learning  and  satisfaction,  and  as  a necessary  component  to  increasing  social  presence  and  sense  of 
community  in  Web-based  courses.  Research  by  Freitas,  Myers,  and  Avtgis  (1998)  sheds  light  on  the 
possible  implementation  of  verbal  immediacy  in  Web-based  courses.  They  found  that  students  enrolled  in 
conventional,  face-to-face  classes  and  those  enrolled  in  a web-based  synchronous  courses  perceived 
differences  in  the  amount  and  quality  of  (instructor)  nonverbal  immediacy,  but  not  in  verbal  immediate 
behaviors. 


Method 

Narrative  inquiry,  a type  of  qualitative  research,  was  used.  Online  instructors’  experiences  are  a 
main  resource  for  this  study.  The  major  reason  to  use  experiences  as  a major  resource  is  to  seek  out  missing 
connection  between  academic  research  and  professional  practice.  Such  researchers  as  Schon  (1987), 

Lynton  (1984),  and  Brooks(1967)  argue  that  academic  research  is  not  only  separate  from  professional 
practice  but  has  been  increasingly  caught  up  in  its  own  agenda,  divergent  from  the  needs  and  interests  of 


227 


professional  practitioners.  To  find  an  answer  to  the  problem  of  gaps  between  practice  and  theory,  I adopted 
Schon’s  (1987)  suggestion  for  research,  which  led  this  study  to  start  not  by  asking  how  to  make  better  use 
of  research-based  knowledge  but  by  asking  what  we  can  learn  from  a careful  examination  of  the 
competence  by  which  practitioners  such  as  online  instructors  actually  handle  their  practice.  Following  John 
Dewey  (1938a),  Schon  (1983),  and  Clandinin  and  Connelly  (2000),  I focus  on  personal  practical 
knowledge  of  online  instructors  reflected  in  their  experiences  and  embedded  in  their  practice,  in  the  hope 
that  it  will  inform  practice  and  theory. 

The  primary  participants  in  this  study  included  three  online  instructors  teaching  undergraduates 
and  graduates  at  a large,  urban  four-year  university  in  Houston,  Texas.  The  secondary  participants  were 
volunteer  students  of  the  online  instructor  participants.  One  instructor,  Dr.  Parker,  was  a male  Caucasian 
associate  professor  who  had  taught  English  classes  for  33  years  and  4 years  in  online  settings.  The  second 
instructor.  Dr.  Raji,  was  a female  Indian  associate  professor  of  Accounting  in  the  College  of  Business.  Dr. 
Raji  had  been  teaching  for  25  years  and  for  4 years  in  online  environments.  Dr.  Williams  was  a male 
Caucasian  associate  professor  of  Mathematics  with  more  than  20  years  of  experience  in  higher  education; 
he  had  taught  online  classes  for  four  semesters.  Pseudonyms  were  used  for  the  sake  of  participants’ 
anonymity.  This  study  used  the  interview  method  combined  with  conversations  and  observations.  1 
observed  online  classrooms  of  the  three  instructors  for  the  duration  of  the  project  and  had  occasional  face- 
to-face  meetings.  This  eight-month  study  was  conducted  from  October  2004  to  May  2005. 

Result 

Four  themes  emerged  from  narratives  of  online  instructors  and  their  students.  As  I was  threading 
and  interpreting  the  narratives  in  findings,  I noticed  that  themes  of  narratives  from  each  participant  fell  into 
four  categories.  To  find  a simple  way  to  present  these  interwoven  themes  embedded  in  practice,  I employed 
the  four  integral  quadrants  for  instructional  immediacy  shown  below  (Figure.  1).  These  four  categories 
emerged  as  online  instructors  negotiated  and  conceptualized  their  understanding  of  instructional  immediacy 
in  Web-based  contexts.  I found  that  effective  teaching  practices  which  enhance  instructional  immediacy  do 
not  only  come  from  interaction  and  communication  but  also  from  integral  use  of  other  elements  of  Web- 
based  instruction  such  as  technology  adoption  and  course  presentation/organization.  It  is  also  important  to 
see  that  immediacy  behaviors  can  be  derived  from  not  only  affective  interactions  but  also  cognitive 
interactions.  In  the  threaded  stories  of  my  participants,  1 found  that  there  are  two  different  kinds  of 
interaction  in  online  communication:  affective  and  cognitive.  Affective  interaction  refers  to  the  interaction 
that  is  executed  for,  and  results  from,  personal  attention  and  caring  responses.  Cognitive  interaction  refers 
to  the  interaction  that  is  conducted  for,  and  results  in,  the  achievement  of  the  content  of  subject  matter. 
Affective  (social)  interaction  opens  the  door  to  building  and  maintaining  the  relationships  between  students 
and  the  instructor;  it  can  affect  the  level  of  achievement  of  students  creating  a strong  and  engaging  learning 
community.  However,  cognitive  interaction  also  contributes  to  the  building  of  relationship  with  students. 

As  long  as  students  can  feel  the  instructors  are  out  there  listening  to  them,  students  feel  the  presence  of  the 
instructor  and  feel  higher  satisfaction  toward  the  instructors’  attitudes.  When  students  receive  prompt, 
detailed  feedback  and  comments,  they  consider  instructors’  actions  as  caring.  Learning  and  teaching  in  a 
Web-based  environment  is  so  static  on  the  computer  screen  that  simple  feedback  from  the  instructor 
appeals  to  students’  perceptions  of  teaching  presence,  that  the  instructor  is  out  there  listening  to  students. 
Although  it  is  difficult  to  express  nonverbal  and  verbal  immediacy  cues  in  a Web-based  classroom,  the 
evidence  suggests  that  online  students  appreciate  any  form  of  instructor  action  presented  during  course 
activities,  because  online  students  expect  less  interaction — they  anticipate  the  instructor  will  be  less  present 
than  in  face-to-face  settings.  Another  phenomenon  found  in  this  study  is  that  the  immediacy  approaches 
that  instructors  chose  were  dependent  upon  the  size  and  contents  of  classes.  With  bigger  size  courses, 
instructors  opted  to  choose  collective  activities  rather  than  individual  activities  due  to  the  limitation  of  time. 


228 


Figure  1.  Integral  Quadrants  for  Instructional  Immediacy 


I.  Affective  Interaction 

Individual-Interior 

Inter-subjective 

II.  Cognitive  Interaction 

Individual-Interior 

Inter-subjective 

III.  Technology  Adoption 

Collective-Exterior 

Objective 

IV.  Course  Presentation/Organization 

Collective-Exterior 

Objective 

/.  Affective  Interaction 

Lived  strategy  one:  personal  attention.  The  English  professor.  Dr.  Parker,  found  a key  drive  of 
affective  online  interaction  in  personal  attention  and  acts  of  caring  (Noddings,  1984).  He  found  that  the 
students  were  “right  there”  with  him  when  he  approached  students  with  personal  attention. 

DR.  PARKER:  A woman  talked  about  her  baby  being  home  sick  for  the  day  and  so  she  is  working 
around  it.  So  I’ll  just  say  something  like,  ‘Well  I hope  your  baby  gets  better  and  take  advantage  of 
the  time  you  are  at  home,’  and  things  like  that.  Someone  said  they  got  fired  from  their  job  so  they 
got  more  time  to  devote  to  the  class.  You  know,  I talk  about  possibilities  for  new  jobs  for  him  and 
things  like  that.  They  talk  about  their  situation We  are  just  working  together. 

Dr.  Parker  expressed  his  sympathy,  comforting  students,  adding  caring,  warm  comments,  and 
offering  advice  in  his  interaction  with  students.  To  build  his  personalized  interaction,  he  started  by 
collecting  students’  personal  information  from  the  beginning  of  the  semester,  through  asking  students  to 
write  self-introductions,  then  by  revisiting  self-introductions  as  students  approached  him  with  their  personal 
issues.  Dr.  Parker,  understanding  that  he  could  not  rely  on  physical  presence  as  in  face-to-face  classrooms, 
needed  to  seek  out  alternative  “ways  to  make  personal  connections  with  students  in  his  class.”  Dr.  Parker’s 
personal  attention  created  a real  sense  of  a virtual  instructor  for  students  so  his  students  claimed  that  “ he  is 
real.” 

Lived  strategy  two:  facilitating  telephone  use.  To  get  students  involved  in  the  class  and  to  get 
them  back  into  good  study  habits,  Dr.  Williams  sometimes  chose  the  telephone  option.  He  told  students  in 
their  orientation  session  that  they  could  call  him  if  they  needed  to,  thus  providing  a more  accessible  way  to 
contact  him.  He  mentioned  that  the  telephone  conversations  provided  answers  and  a certain  atmosphere  that 
email  was  not  able  to  offer.  He  said  students  who  were  in  desperate  need  of  assistance  called  him.  His 
phone  conversations  with  students  mostly  related  to  the  difficulty  of  understanding  the  material  and  anxiety 
about  failing  the  course. 

II.  Cognitive  interaction 

Lived  strategy  three:  promptly  responding  to  students  ’ needs.  Dr.  Parker  learned  how  it  was 
important  for  students  to  receive  feedback  from  the  instructors  promptly.  His  goal  was  a turn-around  time 
of  12  hours,  since  he  was  concerned  that  students  were  waiting  for  his  replies: 

I try  to  get  back  to  them  within  twelve  hours. . ..I  make  a religious  effort  to  do  that  but  sometimes, 
let’s  say  I look  at  it  at  five  o’clock  in  the  evening  and  the  next  time  I look  at  it  is  nine  o’clock  in 
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the  morning  so  that’s  just  not  twelve  hours.  What  I’m  trying  to  say  is  that  twice  a day. . .that 
doesn’t  mean  that  at  nine  o’clock  in  the  evening  and  then  at  noon.  But  twice  a day,  separated  from 
each  other.  I try  to  respond  twice  a day. 

He  understood  that  prompt  feedback  was  a way  that  he  could  show  his  care  for  students.  Because 
Dr.  Parker  exchanged  emails  as  his  primary  mode  of  interaction  with  students,  he  sought  to  narrow  the  time 
it  took  for  him  to  respond  to  them.  One  of  Dr.  Parker’s  students  mentioned  that  her  teacher  always 
attended  to  his  students’  activities,  sending  emails  to  remind  them  of  their  missed  tasks.  “If  you  don’t  turn 
in  your  weekly  journal,  then  he’ll  email  you.  Whereas  other  online  classes,  they  don’t  care  whether  you  log 
in  or  not.”  Britney  felt  affinity  for  the  instructor  in  Dr.  Parker’s  class. 

In  contrast,  giving  prompt  feedback  was  a challenge  for  Dr.  Raji,  who  had  130  students.  She  felt  limited 
and  too  overwhelmed  to  answer  all  emails  promptly.  She  could  not  do  other  responsibilities  because  she 
spent  too  much  time  replying  frequently  to  students.  She  decided  to  manage  her  work  schedule  to  be  more 
effective.  She  informed  students  how  long  it  would  take  until  they  got  email;  she  emailed  them  back  every 
other  day,  trying  not  to  have  students  lose  a feeling  of  connection  and  begin  to  feel  lost  in  cyberspace. 

Lived  strategy  four:  facilitating  chat  sessions.  One  of  Dr.  Williams’s  means  to  initiate 
conversation  with  students  was  synchronous  virtual  meetings.  Every  night  at  9 p.m.,  he  logged  on  to  the 
online  chat  room  and  waited  for  his  students  to  enter,  ask  questions,  and  talk  to  him.  He  wanted  to  make 
sure  that  if  somebody  out  there  needed  help,  they  could  approach  him  without  delay.  Students  knew  he 
would  be  there.  If  students  needed  help,  they  would  know  where  to  go  find  him.  There  were  always 
regulars  who  took  advantage  of  the  chat  session,  as  well  as  those  who  never  used  the  resources  offered  by 
the  instructor.  Dr.  Williams  was  oftentimes  alone,  with  no  one  showing  up  to  chat  toward  the  end  of 
semester.  At  the  same  time,  he  retained  the  chat  session  space  because  he  wanted  his  students  to  get  the 
most  from  his  class. 

When  I asked  one  of  Dr.  Williams’  students  about  the  chat  sessions,  I found  some  very  interesting 
phenomena.  My  impression  of  Dr.  Williams’s  synchronous  meeting  was  that  it  was  not  a success  story,  that 
students  did  not  find  value  in  it.  After  interviewing  one  of  his  students,  I found  that  was  misconception. 
Interestingly  enough,  though  Maggie  attended  once  in  the  semester,  she  found  the  synchronous  chat  session 
informative.  She  explained  that  the  chat  session  time  did  not  align  with  her  own  schedule  with  her  kids. 
However,  she  logged  on  later  when  everyone  else  had  left,  and  checked  on  what  others  were  talking 
about — perhaps  they  were  asking  the  same  questions  as  she.  I asked  about  the  value  of  the  chat  session,  and 
she  suggested  its  continuance.  Maggie,  as  1 understand  it,  had  experienced  vicarious  immediacy,  feelings  of 
closeness  by  witnessing  interactions  between  one’s  teacher  and  other  students  (LaRose,  Gregg,  & Eastin, 
1999)  in  chat  rooms.  In  Maggie’s  report,  I found  two  different  kinds  of  participation:  visible  participation 
and  invisible  participation.  Students  who  missed  the  chat  session  participated  in  the  synchronous  discussion 
activity  by  logging  on  after  the  session  closed,  and  reading  the  transcript  of  the  sessions. 

Lived  strategy  five:  Initiating  the  contact.  Proactive  where  communication  was  concerned,  Dr. 
Williams  initiated  contact  with  students,  because  online  students  have  been  found  to  be  reticent  in  soliciting 
help.  Dr.  Williams  and  I,  having  had  the  experience  of  being  an  online  learner,  acknowledge  students’ 
reluctance  in  seeking  help.  When  Dr.  Williams  discovered  a student  who  was  struggling,  he  did  not  want  to 
make  the  student  think  he  perceived  him  as  someone  who  was  doing  poorly.  The  sensitive  situation 
deterred  Dr.  Williams  from  attempting  repeated  contacts.  Students  are  laboring  away,  yet  instructors  are 
hesitant  to  approach  students  with  problems  because  of  the  touchy  situation.  However,  he  ensured  that 
students  knew  he  would  be  available  whenever  they  needed  him  through  emails  and  orientations.  One  of 
his  students  acknowledged  his  efforts  at  initiating  contacts  with  his  students.  She  observed  that  he  kept 
track  of  students’  works  and  guided  students  who  were  off  task  in  the  right  direction.  It  is  significant  to  see 
an  online  student  crediting  her  instructor’s  actions,  even  when  the  instructor  was  helping  other  students. 

III.  Technology  Adoption 

Lived  strategy  six:  Learning  students’  learning  styles.  Dr.  Raji  felt  inherent  limitation  in  learning 
through  the  textual  form  in  terms  of  understanding  the  content,  and,  in  turn,  she  employed  the  technology 
to  provide  better  teaching  presence  with  more  sensory  stimulation.  Dr.  Raji  learned  the  importance  of 
human  elements  in  online  teaching  from  workshops  and  conferences,  and  agreed  on  the  effectiveness  of 
humanized  online  classes.  She  tried  to  enhance  technology  with  the  human  elements  by  inserting  her  voice 
into  her  presentation  slides.  She  wanted  students  to  understand  better  with  supplemental  materials  and  to 
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hear  the  “feeling”  of  her  instruction.  Students  without  discipline  did  not  take  merits  from  the  resources 
already  messaged  and  predigested.  She  felt  disappointed  when  she  saw  that  some  students  never  listened  to 
her  voice  lecture,  in  which  she  had  invested  a great  deal  of  her  energy  and  time  in  order  to  promote  the 
better  understanding  of  materials.  In  addition,  students  with  less  advanced  technology  were  prevented  from 
benefiting  from  her  endeavors.  It  seemed  that  these  online  students  followed  the  principle  of  economy  in 
learning:  online  students  who  were  task-oriented  sought  to  achieve  the  best  grades  with  the  least  effort. 
There  were  several  assumptions  to  examine  in  interpreting  this  phenomenon.  Some  students  in  the  course 
did  not  find  time  to  sit  and  listen  to  the  explanations.  It  was  also  the  question  of  adjustment  in  their  learning 
styles.  One  habit  of  online  students  was  to  download  course  materials,  and  read  them  at  their  own 
convenience.  She  came  to  understand  that  the  previously  developed  student  learning  style  with  written 
formats  was  more  comfortable  to  deal  with  than  watching  PowerPoint  presentation.  Also  the  written-format 
tutorials  are  more  convenient  to  glimpse  over  in  their  entirety.  Dr.  Raji  learned  that  sometimes  empowering 
the  traditional  way  of  learning  would  be  superior  to  the  innovative  way  of  instruction.  This  phenomenon 
raised  the  issue  of  online  educators’  tendency  to  rely  on  the  capacity  of  innovative  technology  which  may 
not  fit  in  the  student  developed  learning  styles. 

IV.  Course  Presentation/Organization 

Lived  strategy  seven:  Visual  Channel  of  Personalization:  “Virtual  Physical  Objects.  ” Dr.  Parker 
has  provided  the  great  number  of  visual  channels — his  homepage,  his  welcoming  message  page,  his 
pictures,  and  his  introduction  letter  disclose  himself  and  provide  his  self-image  to  student  for  the  provision 
of  socio-emotional  materials  (Short,  Williams,  & Christie,  1976)  which  lack  in  Web-based  learning  and,  in 
turn,  result  in  increases  of  the  information  available  about  the  instructor’s  self-image,  attitudes,  moods,  and 
reactions  (Short  et  ah,  1976).  He  posted  a picture  of  himself  mountain  climbing,  taken  when  he  taught  the 
Semester  at  Sea  in  his  welcoming  page  to  show  students  that  he  was  an  active  man,  not  a bookworm  in 
university  office  surrounded  by  the  bookshelves.  His  picture  plays  a role  to  create  his  “electronic  self- 
image  and  personality,”  of  being  adventurous  and  active,.  To  be  more  personalized,  Dr.  Parker’s  adds 
personal  message  below  the  picture. 

Hi!  That  is  me  on  the  right,  Dr.  Parker,  your  professor  for  this  class.  Iam  not  actually  waving  to 
you  but  to  some  other  students  who  reached  the  summit  of  the  Avacha  volcano  ahead  of  me.  That 
was  one  of  my  adventures  this  summer  when  I taught  on  Semester  at  Sea.  During  the  semester,  I 
will  add  other  images  to  give  you  a sense  of  my  experience  and  Semester  at  Sea  in  general  (Dr 
Parker’s  Welcoming  Message). 

The  style  of  his  greeting  creates  the  feeling  of  being  welcomed;  he  establishes  a warm  and  inviting 
presence  (Palloff  & Pratt,  2003).  These  visual  cues  have  been  ways  to  form,  build,  or  maintain  the 
relationship  between  interactants  (Short  et  ah,  1976).  Through  the  use  of  a humorous  picture,  his  homepage 
invites  viewers  to  navigate  more  of  the  pages. 

Lived  strategy  eight:  Written  Channel  of  Personalization:  “I,  ” “You,  ” and  “Narrative.  ” It  is 
common  that  an  online  presentation  style  should  be  a literary  style  rather  than  a dialogic  one.  Dr.  Parker, 
however,  explained  that  online  classes  should  have  features  of  the  conventional  classroom,  where  the 
instructors  present  their  instruction  in  a conversational  way.  Instead,  he  adopts  the  conversational  style  in 
his  writings  for  class  Web  sites,  using  “I,”  “you,”  and  “narrative.”  The  followings  are  the  examples  of 
dialogic  style  in  his  “Keeping  a Journal  Subject”  page: 

Generally  speaking  you  should  write  about  on-line  instruction,  participation,  and  issues 

relating  to  reading  and  studying  Shakespeare.  You  might  relate  positive  aspects  such  as  things  you 
have  learned,  or  what  you  have  discovered  as  a result  of  this  class.  You  will  probably  also  have 
some  frustrating  experiences  with  the  class  and  with  Shakespeare,  and  I would  like  to  hear  of 
those  as  well.  Or  you  may  have  some  general  or  technical  suggestions  for  the  class  which  I hope 
you  will  pass  along. ... 

According  to  Lee  (2004),  due  to  the  subjective  nature  of  the  perception  process,  people  can  create 
the  sense  of  presence  based  solely  on  cognitive  stimuli  for  imagination  without  receiving  any  direct  sensory 
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stimuli.  Dr.  Parker  provides  “written  narrative”  as  an  example  of  cognitive  stimuli.  He  provides 
information  on  how  to  complete  assignments  within  a conversation,  possible  situations  students  might 
undergo  when  taking  an  online  class,  and  instruction  how  not  to  fail  to  complete  assignments.  Another 
interesting  aspect  is  that  he  creates  “social  space”  (Palloff  & Pratt,  2003),  where  students  can  express  their 
frustration  and  feelings  to  instructors,  saying  “You  will  probably  also  have  some  frustrating  experiences 
with  the  class  and  with  Shakespeare,  and  I would  like  to  hear  of  those  as  well.”  Furthermore,  he  lets  them 
know  he  is  ready  to  receive  suggestions  from  them:  “You  may  have  some  general  or  technical  suggestions 
for  the  class  which  I hope  you  will  pass  along.” 

Lived  strategy  nine:  Disciplining  students.  Despite  limitations,  classroom  instruction  works.  Web- 
based  instruction  is  new,  different,  and  crude  (Horton,  2000).  To  succeed  with  Web-based  instruction, 
online  instructors  had  to  find  ways  to  work  with  students  who  have  been  trained  by  traditional  ways  of 
learning.  Dr  Parker  found  that  one  of  the  mistakes  students  repeatedly  made  was  that  students  miss  reading 
notices  posted  in  multiple  places  throughout  the  Web  site  because  they  do  not  have  peers  to  remind  them 
about  class  procedures,  or  professors  who  repeat  instructions  during  the  next  class.  Thus,  Dr.  Parker 
encourages  his  students  to  be  more  alert  than  they  are  in  a traditional  classroom,  and  walks  students 
through  the  requirements,  detailing  what  is  expected.  This  is  a step  toward  establishing  routines  of  the 
class,  a step  toward  the  structured  discipline  for  students  who  might  get  lost  in  this  crude,  new 
environment.  He  confesses  that  occasionally  he  does  not  repeat  himself,  instead  he  asks  students  to  remind 
him  and  ask  questions  if  anything  is  ambiguous.  It  opens  a more  informal,  mistake-allowed  atmosphere.  Dr. 
Parker’s  weekly  message  section  becomes  an  announcing  board  where  students  gradually  establish  their 
habits  and  visit  to  pace  their  learning. 

Lived  strategy’  ten:  Creating  momentum  in  communication.  We  learn  how  to  behave  in  situated 
contexts  and  a given  culture.  When  the  momentum  of  interaction  is  established,  the  better  opportunity  is 
given  to  the  relationship  between  instructors  and  students.  Online  instructors  came  to  know  their  students 
through  their  way  of  discussion  and  through  their  assignments.  More  frequent  interaction  framed  the 
learning  community  to  communicate  one  another  specific,  personal,  and  contextual.  Dr.  Parker  developed 
momentum  in  communication  by  supporting  a discussion  board  and  journal  writing.  The  discussion  board 
and  journal  writing  developed  students’  habits  to  execute  the  learning  events  while  they  were  participating 
in  the  events  on  a regular  basis.  This  can  be  seen  as  the  gradual  internalization  of  processes  initially  shared 
between  participants  of  the  learning  activity  (Chang-Wells  & Wells,  1993).  Keeping  journal  entries  on  a 
weekly  basis  and  responding  to  the  discussion  topics  on  discussion  board — these  activities  function  to 
immerse  students  in  a routine  of  communication.  Students  write  journal  entries  of  150  words,  and  turn  them 
in  every  Friday.  The  journaling  activity  provided  insight  for  Dr.  Parker:  he  knows  the  students’ 
characteristics  better;  students  improve  their  writing  and  thinking;  students  develop  habits  of  writing;  also, 
students  finish  the  reading  assignments  in  order  to  complete  their  journaling. 

Dr.  Raji  also  found  a relationship  between  the  level  of  student  interaction  and  their  subsequent 
achievement.  She  found  the  link  between  the  frequency  of  the  interaction  and  student  academic 
achievement.  Facilitating  the  discussion  board  as  a place  of  forum  and  problem  solving,  Dr.  Raji  had  been 
able  to  make  interaction  flow  in  her  online  class.  With  organized  monitoring  mechanism,  a teaching 
assistant  or  Dr.  Raji  responded  in  a timely  manner  to  posted  questions.  In  doing  so,  students  would  know 
this  was  the  place  they  could  turn  to  any  time  they  had  questions.  The  instructor  could  read  what  material 
caused  problems,  which  learners  needed  more  explanation,  who  was  active,  who  was  reluctant  or  absent, 
what  their  expectations  were,  and  so  on.  It  made  the  online  classroom  more  visible,  which  made  effective 
intervention  possible. 

Dr.  Williams  notices  that  nontraditional  students  who  have  not  been  exposed  to  online 
environments  or  have  not  taken  traditional  courses  for  a while  lack  the  sense  of  rigor  and  intensity 
necessary  of  online  students.  Students  enrolled  in  face-to-face  classes  have  more  opportunities  to  feel 
pressured  to  finish  their  work  by  hearing  the  instructor  repeat  reminders  aloud,  and  by  observing  their  peers 
discuss  and  prepare  assignments.  Talking  to  peers  about  the  assignment  and  their  work  process  also 
provides  a sense  of  urgency  to  traditional  students  in  regular  classes.  This  is  the  pressure  arising  from  their 
learning  communities.  According  to  Dr.  Williams,  online  students  lack  the  support  from  instructors’ 
physical  messages  and  the  pressure  from  learning  communities. 

Dr.  Williams  mentioned  the  use  of  a session  in  a chat-room,  which  works  as  a virtual  synchronous 
classroom.  He  explained: 

Well,  one  solution  is  to  make  them  do  something  at  a certain  time.  For  example,  make  them  enter 

the  chat  room  for  a class  discussion  at  a certain  time  so  that  it’s  like  being  in  class.  That  way  they 
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get  involved.  And  tell  them  that  you’re  going  to  notice  if  they’re  not  there,  you  know,  and  it  may 
affect  their  grade.  That  way  it’s  more  like  being  responsible  for  being  in  class  at  a certain  time, 
and  so  they  have  a little  more — there’s  a little  more  urgency,  1 think,  for  them.  That’s  one  way  to 
do  it. 

Dr.  Williams  uses  the  chat  session  to  impress  on  his  students  the  habit  of  attendance;  he  also 
monitors  students  to  see  if  they  are  keeping  up  with  the  learning  events.  He  believes  these  kinds  of  actions 
possibly  create  a sense  of  urgency  about  completing  assignments. 

Discussion 

The  main  findings  of  this  study  suggest  that  online  instructor  immediacy  could  and  should  be 
enhanced  in  the  areas  of  instructional  design  of  courses,  interaction  with  students,  learning  activities,  and 
learning  materials.  In  addition,  the  methods  of  creating  online  instructor  immediacy  should  be  adapted  to 
the  characteristics  of  Web-based  culture,  online  students,  content  area,  and  students’  learning  styles. 
Through  the  presentation  and  analysis  of  the  online  practitioners’  threaded  stories,  this  study  illustrates  the 
instructors’  approaches  to  enhance  instructional  immediacy:  synchronous  virtual  meetings  (e.g.,  chat 
sessions)  and  asynchronous  virtual  meetings  (e.g.,  discussion  board),  real-time  handwriting  with  voice-over 
presentation,  the  images  of  the  instructors,  caring  interactions,  prompt  feedback,  emails,  instructor-lead 
instruction,  and  the  like.  Online  instructors  tell  us  that  online  instructor  immediacy  in  a virtual  world  is  not 
far  from  where  we  are.  As  he  made  clear,  he  is  not  executing  an  innovative,  cutting-edge  way  of  teaching 
online,  but  a more  personalized,  structured,  and  caring  way  of  “working  together.”  He  found  that  the  online 
learning  communities  were  smoothly  moving  toward  success  when  we  learned  how  to  “work  together.” 
Web-based  environments,  where  the  instruction  is  delivered  electronically,  are  new  to  both 
learning  and  teaching  parties  so  that  Web-based  environments  call  for  human  adjustment.  Dr.  Williams, 
who  has  been  a traditional  teacher  for  more  than  20  years,  relied  on  reading  and  interpreting  students’ 
visual  and  verbal  cues.  Online,  however,  he  could  not  receive  any  of  those  cues.  The  lack  of  physical  cues 
caused  Dr.  Williams  to  fail  to  determine  to  what  extent  students  understand  his  instruction.  As  a result,  he 
felt  frustrated  and  oblivious  of  the  state  of  their  learning.  Therefore,  he  decided  to  understand  students  by 
monitoring  the  chat  session  where  students  synchronously  communicate  like  in  a class.  His  methods  of 
understanding  students  have  since  been  adjusted.  The  adjustment  was  also  required  for  students’  learning. 
However,  students’  study  habits  oftentimes  do  not  translate  well  in  the  new  environment.  Even  though  Dr. 
Raji  offered  beneficial  voice-over  PowerPoint  presentations,  not  many  students  took  advantage  of  the 
course  resource  provided.  Students,  it  appears,  do  not  easily  learn  how  to  adjust  their  learning  style  and 
make  the  transition  from  a traditional  structure  to  a “free”  and  self-paced  structure  (Shih,  2002). 

Online  educators  should  recognize  that  the  online  environment  calls  for  more  presence  from  the 
instructors.  It  is  easy  to  fall  into  the  illusion  that  expensive  technology  could  solve  all  pedagogical 
dilemmas  and  lead  us  to  instructional  success.  However,  for  online  students  who  are  mostly  task-oriented, 
who  are  not  up-to-date  on  technology,  and  who  lack  the  time  to  attend  class  on  campus,  the  use  of  time- 
consuming  technology  is  not  appealing.  Well-organized  mechanism  of  classes  makes  students  keep  up  with 
the  course  modules  and  forces  them  to  leam;  they  successfully  address  students’  abilities  to  procrastinate. 

Dr.  Parker  repeatedly  emphasizes  that  the  class  size  is  critical  to  the  success  of  online  education. 

He  asserts  that  if  an  online  instructor  has  more  than  30  students,  there  is  no  way  he/she  can  create  a 
personalized  online  class  with  prompt  feedback,  a caring  relationship,  and  knowledge  of  the  students,  all  of 
which  are  key  to  building  a learning  community  and  online  instructor  immediacy. 

I learned  that  online  instructors  needed  time  to  reflect  on  their  practice.  After  finishing  my 
interviews  with  participants,  I asked  them  how  my  interview  influenced  their  thinking  or  beliefs  on  online 
teaching.  They  said  my  interview  questions  caused  them  to  think  about  issues  which  never  occurred  to 
them.  Their  effective  teaching  takes  shapes  when  instructors  filter  their  professional  development 
experiences  through  their  reflection  on  mere,  unconscious,  and  tacit  knowledge  in  practice. 

Conclusion 

American  higher  education  has  long  struggled  with  the  “right”  model  for  facilitating  this  kind  of 
connection  between  faculty  and  students.  The  findings  of  this  study  provide  online  educators  with 
instructive  models  of  the  teaching  and  guidelines  for  humanizing  and  enhancing  instructor  immediacy  in  a 
Web-based  environment,  and,  in  turn,  have  helped  bridge  the  gap  between  the  guidelines  for  online 
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teaching  and  the  enactment  of  online  teaching,  by  providing  detailed  narration  of  the  teaching  process. 
O’Donnell  (1998)  states  that  technology  can  be  dehumanizing  and  distancing,  emphasizing  on  the  value  of 
the  personal  contact  that  motivates,  ignites,  and  guides  learning.  Keegan  (1986)  also  claims  that  distance 
education  has  to  compensate  with  non-aural  language.  Many  instructors  and  learners  fear  that  with  Web- 
based  instruction  they  will  lose  the  human  touch  of  classroom  instruction  (Horton,  2000).  There  is  a 
widespread  belief  that  the  lack  of  face-to-face  contact  means  that  distance  learning  is  impersonal  (Kubala, 
1998).  To  address  the  concerns  related  to  dehumanization  and  impersonalization  of  online  education,  the 
findings  in  this  study  offer  both  possibilities  to  act  on  and  issues  that  need  to  be  considered  in  building 
instructional  immediacy  in  online  education.  My  three  online  practitioner  participants  offered  us  pathways 
that  suggest,  but  not  dictate,  better  ways  to  proceed  with  online  education.  Now  it  is  time  to  consider  how 
we,  as  online  educators,  can  learn  from  their  knowledge  derived  form  their  online  teaching  experiences  and 
how  we  can  address  the  issues  with  which  they  struggled,  not  only  at  the  individual  level,  but  also  at  the 
institutional  level. 
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Abstract 

This  study  proposes  a web-based  problem-solving  environment.  Animal  Investigator,  for  improving  students' 
knowledge  generation  in  a scientific  context  and  for  enhancing  students’  problem-solving  skills  in  an  ill-structure 
problem  context.  The  design  of  Animal  Investigator  is  based  on  the  two  cognitive  processes  related  to  the  scientific 
knowledge  generation  for  adolescents:  classification  and  hypothesis  generation.  Particularly,  both  processes  are 
considered  as  important  problem-solving  skills  which  students  need  to  solve  unfamiliar,  real-world  problems.  The 
Animal  Investigator  environment  allows  students  to  be  involved  actively  in  two  training  sessions  and  ill-structured 
problem-based  learning  as  they  personalize  the  learning  context.  To  evaluate  the  design  usability  for  young 
adolescents,  age  twelve,  two  surveys  were  conducted  on  twenty-eight  sixth  grade  students  in  Korea  who  had 
completed  the  classification  of  animals  units  in  the  spring  semester.  The  students  completed  two  surveys  after  using 
the  Animal  Investigator.  Constructivist  Internet-based  Learning  Environment  Survey  for  Science  (CILES-S)  and 
Intrinsic  Motivation  Inventory  (IMI).  Future  directions  include  using  the  Animal  Investigator  for  research  on 
hypothesis  generation  skill  development  in  ill-structured  problem  solving  and  learner's  scientific  knowledge 
generation  skill  development. 


Introduction 

Over  the  past  decade,  we  have  witnessed  a dramatic  change  in  opportunities  for  computer-based  learning  in 
education  theory  and  in  school  (Crooks,  Verdi,  & White,  2005).  We  have  developed  theoretical  foundations  to 
support  the  development  of  computer-based  learning  environments.  One  of  the  cruxes  in  these  theories  in 
developing  a tool  is  constructivist  perspective  toward  knowledge  construction.  Constructivist  knowledge  is 
understood  as  actively  being  built  up  by  the  subject,  who  uses  adaptive  cognition  in  the  external  world,  not  the 
discovery  of  an  objective  external  reality  (von  Glasersfeld,  1988);  that  is,  knowledge  is  made,  not  acquired  (Phillips, 
2000).  A constructivist  view  has  led  to  a reconsideration  of  epistemology  for  teaching  and  for  learning  scientific 
knowledge  acquisition  in  science  education  (Lawson,  2003).  For  example,  Staver  (1998)  stated,  "For  constructivists, 
observations,  objects,  events,  data,  laws,  and  theory  do  not  exist  independent  of  observers.  The  lawful  and  certain 
nature  of  natural  phenomena  are  properties  of  us,  those  who  describe,  not  of  nature,  what  is  described"  (p.  503). 
From  this  perspective,  knowledge  is  temporary,  developmental,  non-objective,  internally  constructed,  and  social  and 
culturally  mediated  (Fosnot,  1996).  These  features  reflect  the  process  of  knowledge  construction  and  acquisition.  In 
science  education,  it  is  fundamental  to  progress  from  specific  hypotheses  generated  during  observation  and  research 
of  an  observed  phenomenon  to  informed  theories  in  order  to  construct  scientific  knowledge.  Given  any  science 
curriculum,  the  hypothesis  generation  must  be  introduced  as  a crucial  science-learning  activity  (Klahr  & Dunbar, 
1988;  Lawson,  1995).  The  other  scientific  process  needed  to  generate  knowledge  is  classification,  which  means  to 
identify  patterns  to  make  a decision  about  a group  of  correlated  features.  Classification  depends  upon  the 
characteristics  of  its  constitutive  elements  and  the  various  ways  in  which  they  interact  (Butos  & Koppl,  2003).  In  a 
classification  process  to  develop  knowledge,  a critical  strategy  is  a hypothesis  generation  (Bruner,  Goodnow,  & 
Austin,  1956).  Both  strategies  for  generating  knowledge  are  also  considered  as  fundamental  strategies  to  solve 
problems,  which  students  need  to  use  in  order  to  solve  unfamiliar  and  real-world  problems.  In  terms  of  both 
strategies  that  generate  knowledge,  we  may  strengthen  a learner’s  problem-solving  skills  in  an  ill-structured  problem 
context.  To  support  this  research,  a computer-based  learning  environment  (CBLE)  is  considered  as  an  environment 
in  which  students  can  participate  in  challenging  problem-solving.  CBLE  have  the  potential  to  enhance  learning 
about  complex  topics  (Azebedo,  2005;  Azebedo,  et  al,  2005).  In  order  for  CBLE  to  facilitate  a student’s  learning, 
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Azebedo  (2005)  asserts  that  these  CBLEs  have  to  have  effective  self-regulatory  skills  in  order  to  control  the  learning 
phase  and  time,  to  access  multiple  representations  of  information,  and  to  make  choices  about  non-linear  multiple 
representations  (Azebedo  & Cromley,  2004).  The  purpose  of  this  research  is  to  design  and  implement  a CBLE, 
Animal  Investigator  (AI),  and  to  enhance  scientific  learning  processes  across  various  contexts. 

Design  of  the  Animal  Investigator 


Program  description 

"Animal  Investigator"  is  a web-based  learning  module.  The  " Animal  Investigator"  provides  young-adolescents-level 
(12-15  years)  students  a guide  for  the  purpose  of  supporting  learner's  knowledge  generation  activities  for  enhancing 
strategies:  hypothesis  generation  and  rationalization  in  classification.  Animal  Investigator  also  created  to  encourage 
students  to  become  independent  learners  and  to  develop  new  learning  strategies  by  their  own  regulated  function, 
metacognitive  strategies. 

In  the  main  menu,  there  are  four  phases  of  Animal  Investigator  module:  "Functions,"  "Investigation," 
"Classification,"  and  "Problem  solving." 

The  Animal  Investigator  starts  with  a letter  as  an  example  in  order  to  understand  the  goal  of  the  program  (Figure  1). 
Then,  the  learners  go  to  a short  tutorial  and  manipulate  buttons  at  the  bottom,  which  is  a main  menu  for  all  of  the 
steps  of  Animal  Investigator,  including  system  help,  a scaffold  for  facilitating  metacognitive  thinking  process,  a 
scaffold  for  supporting  their  training  process  as  a function  of  expert  support,  a notebook  to  review  learners’  memos 
and  classification  outputs,  a scratch  pad  for  writing  their  thoughts,  ideas,  and  information,  and  a menu  program 
menu  (Figure  2).  In  the  letter,  the  learner's  name  is  shown  on  the  screen,  which  will  increase  their  involvement  and 
absorption.  Cordova  and  Lepper  (1996)  argue  that  a display  of  the  user's  name  on  the  screen  can  personalize  the 
learning  context  and  increase  his  or  her  motivation. 

Two  of  the  trainings  for  the  students  who  participate  in  the  program  involve  the  teaching  of  metacognitive  strategies 
during  locating,  analyzing,  selecting,  evaluating,  and  integrating  information  from  sources,  and  include  the  teaching 
of  strategies  to  improve  hypothesis  generation  in  classification,  because  both  competencies  are  considered  as 
important  problem-solving  skills  in  an  ill-structured  problem-solving  environment.  After  learning  the  two  main 
training  sessions,  students  then  solve  an  ill- structured  problem  in  the  problem-based  learning  stage. 

Stage  1:  Investigation 

In  the  first  training,  "Investigation,"  learners  begin  seeking  and  using  given  information  through  the  training 
situation.  Learners  acquire  information  from  the  text  and  pigeon  photos  in  order  to  answer  open-ended  questions. 
Learners  who  follow  the  functions  module  select  the  next  phase  from  the  main  menu  at  the  top  to  start  their  first 
training.  Before  starting  the  main  training,  the  question  of  describing  a bird  is  given  for  the  learner  to  activate  their 
prior  knowledge.  The  learner  is  asked  three  questions  to  activate  her  relevant  knowledge.  For  example,  one  question 
may  be  as  "describe  a bird  precisely  you  remember."  When  learners  submit  their  answers  by  clicking  a gray  submit 
and  begin  button,  the  main  screen  of  the  first  training  is  presented  in  a linear  arrangement,  and  learners  can  access 
each  button  in  a nonlinear  manner  to  find  an  appropriate  explanation  towards  receiving  new  questions.  There  are 
seven  chunks  of  information  about  a pigeon  family.  Learners  can  click  buttons,  which  are  information  chunks  based 
on  their  interests.  There  are  no  required  numbers  of  correct  answers,  but  they  need  to  seek  and  to  analyze  the 
appropriateness  of  the  questions  in  order  to  rationalize  their  thoughts  (Figure  3).  From  this  training,  students  begin  to 
use  cognitive  facilitators,  the  scratch  pad,  the  notebook,  and  metacognitive  scaffolds  and  strategic  scaffolds  buttons 
at  the  bottom.  In  using  these  facilitators,  learners  in  both  training  sessions,  can  utilize  their  functions  to  help 
cognitive  processes  like  memorizing,  reviewing,  summarizing,  comparing,  hypothesizing,  and  making  decisions.  To 
overcome  young  adolescents'  limited  cognitive  capacities  such  as  working  memory  and  prior  knowledge,  it  is 
necessary  to  facilitate  their  learning  process  by  these  means. 

Stage  2:  Classification 

In  the  second  training,  learners  are  faced  with  a more  complicated  problem  environment  characterized  by 
uncertainty.  There  are  eight  sparrows  which  are  named  "bird."  There  are  no  clues  by  which  6th  grade  students  can 
identify  the  exact  name  of  the  birds.  Learners  can  analyze  the  given  materials  according  to  six  parts  of  information, 
such  as  general  identification,  distribution,  measurements,  leg  features,  bill  features,  and  tail  features.  Eight 
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sparrows  from  four  species,  Calcarius  Mccownii  (bird  1),  Calcarius  Pictus  (bird  2),  Emberiza  leucocephalos  (bird  3), 
Emberiza  Citrinella  (bird  4),  Zonotrichia  Albicollis  (bird  5),  Zonotrichia  Atricapilla  (bird  6),  Pipilo  Chlomrus  (bird 
7),  and  Pipilo  Erythrophthalmus  (bird  7),  have  distinctive  attributes  among  the  sparrow  family  (Byers,  Curson,  & 
Olsson,  1995;  Beadle  & Rising,  2002).  Before  beginning  this  training,  students  participate  in  the  tutorial  session  to 
glance  at  processes  and  facilitators  related  to  the  training.  In  order  to  classify  birds,  students  must  engage  in  a variety 
of  materials.  They  should  learn  about  each  bird  from  the  materials,  and  should  search  them  for  possible  criteria 
which  overlap  with  the  features  of  birds.  Students  gather  evidence  to  support  their  possible  criterion  by  exploring  the 
materials  (Figure  6).  Finally,  when  students  are  ready  to  allocate  birds  in  terms  of  their  criterion,  they  will  type  their 
criterion  (called  “rules”  for  the  students)  and  rationalize  their  criterion  using  the  location  of  evidence,  which  is 
located  at  the  top  of  each  material  (Figure  5).  After  rationalizing  the  criterion,  students  type  each  attribute  in  the 
boxes  in  the  middle  such  as  "read  head"  or  "long  hind  craw."  Students  then  start  to  drag  and  drop  each  bird 
according  to  each  attribute.  There  is  no  limit  to  the  generation  of  their  criterion  and  attributes.  Students  can  go  back 
to  the  analysis  phase  at  anytime  to  review  the  target  bird's  information  and  to  observe  their  pictures.  When  students 
are  satisfied  with  their  criterion  and  allocations  of  birds  for  each  attribute  column,  they  click  on  the  "add/save" 
button  to  generate  more  classification  for  the  given  birds.  Whenever  students  need  to  review  their  whole 
classification  outcomes,  rules,  reasons,  and  allocated  birds,  they  can  click  the  "review"  button  to  check  all  at  one 
time  using  the  notebook  facilitator,  which  has  two  functions:  reviewing  memos  and  classifying  outcomes.  When 
students  complete  their  classification  to  their  satisfaction,  they  click  on  the  "complete"  button.  Students  then  are 
awarded  an  Animal  Investigator  Certificate  from  a virtual  coordinator  (Figure  7). 

Stage  3:  Problem-Based  Learning 

Finally,  students  are  encouraged  to  attend  a mission  as  an  Animal  Investigator.  Students  encounter  a list  of  news 
articles  concerning  a virtual  country,  Atlantis  (Figure  8).  In  order  to  gain  a better  understanding  and  make  a decision 
about  a problem,  students  analyze  articles  and  report  their  thoughts  about  what  they  have  to  know  in  the  mission. 
The  problem  situation  as  the  follows: 

Animal  Investigator's  country  and  the  Atlantis  government  have  full  diplomatic  relations.  Animal 
Investigator ’s  country  is  interested  in  the  future  development  of  oil  and  gas  near  Atlantis  Bay.  Thus,  both 
countries  have  established  an  official  agreement  about  oil  and  gas  extraction.  An  oil  spill  occurred  at 
Atlantis  Bay  because  an  oil  tanker  hit  a reef  in  Atlantis  Bay.  Animal  Investigator's  country  has  decided  to 
dispatch  a special  team  to  help  clean  the  oil  spill  as  well  as  to  help  rescue  animals  harmed  by  the  oil  spill. 
The  role  of  Animal  Investigator  in  this  mission  is  to  help  allocate  animals  harmed  by  the  oil  spill  to 
appropriate  local  animal  shelters. 

There  is  a "message"  button  to  describe  the  problem  statement  for  the  students'  mission  (Figure  9).  Writing  the 
problem  statement  is  encouraged  so  students  will  participate  in  the  ill-structured  problem.  In  this  step,  learners  are 
faced  with  an  ill-structured  problem  that  required  them  to  decide  if  there  is  a problem.  Jonassen  (1997)  asserts  that 
the  ill-structured  problem  may  not  appear  directly  or  may  be  hidden.  The  uncertainty  of  an  ill-structured  problem 
causes  difficulty  for  the  learner  solving  the  problem.  In  terms  of  submitting  their  response  about  asking  what 
mission  is  from  their  advisor,  their  mission  begins  by  their  advisor  informing  them,  and  by  the  program  showing 
icon  bars  (Figure  10).  From  this  moment,  they  play  as  an  investigator  who  examines  harmed  animals.  As  with  the 
previous  training,  students  meet  with  anonymous  animal  - a sea  turtle,  a bottlenose  dolphin,  and  a sea  gull  - without 
their  names  appearing  on  the  screen.  These  are  the  animals  assumed  covered  with  spilled  oil  in  Atlantis  Bay.  The 
Animal  Investigator  starts  to  search  for  information  using  the  animal  information  button  (Figure  11).  Ultimately  their 
goal  is  to  allocate  each  animal  to  local  animal  shelters  according  to  each  animal's  conditions  and  biological  features 
(Figure  12).  To  analyze  the  conditions,  students  examine  each  status  report  and  features  describing  each  animal.  To 
type  final  decisions  for  each  animal,  students  should  enter  three  parts  of  a response:  animal  identification  and 
rationale,  new  information  from  the  investigation  activity,  and  allocation  of  each  animal  and  rationale  based  on 
information.  In  this  stage,  students  can  also  use  cognitive  facilitators  like  they  do  in  the  previous  trainings. 

In  the  Animal  Investigator , there  are  five  animals,  pigeons,  sparrows  (8  species),  turtles,  dolphins,  and  gulls.  These 
animals  are  selected  from  the  Korea  sixth  grade  textbook  and  two  Texas  sixth  grade  science  textbooks  after 
analyzing  the  unit  on  animal  kingdom  and  classification.  The  problem  context  of  the  oil  spill  case  and  the  animals' 
statuses  were  developed  based  on  three  reports  including  oil  spill  cases  and  animal  damages  due  to  oil  spill  incidents. 
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Figure  1.  Introduction  and  Prompt 


Figure  2.  Program  Tutorial  and  Menu 


Figure  3.  Phase  1:  materials 


Figure  4.  Phase  1:  Scratch  pad 


Figure  7.  Phase  2:  Certificate 


Figure  8.  Phase  3:  e-News  lists 
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Figure  9.  Phase  3:  Message 


Figure  10.  Phase  3:  Investigation 


Figure  11.  Phase  3:  Analysis  of  Animal  Shelters 


Figure  12.  Phase  3:  Allocation  of  Animals 


Design  Principles  of  Animal  Investigator 

Animal  Investigator  (AI)  is  a web-based  problem-solving  environment  relating  to  animal  classification.  Our  basic 
approach  to  supporting  activities  that  generate  scientific  knowledge  can  be  summarized  by  three  program  design 
principles:  First,  we  focused  on  the  classification  process  to  enhance  classifying  animals. 

Second,  we  revolved  around  the  hypothesis  generation  process  as  an  important  strategy  of  classification  in  order  to 
design  facilitating  functions  to  promote  generating  hypotheses  on  “ Animal  Investigator. ” Third,  we  designed  the 
metacognitive  scaffold  to  serve  as  scaffolding  agents.  In  addition,  there  are  more  design  features  as  follows: 

• Making  scaffolds  so  learners  check  the  problem  solving  process: 

• Making  the  most  iterative  interface  for  young  adolescents:  the  iterative  structure  is  a design  component  to 

be  considered  in  the  web-based  learning  environment  in  order  to  decrease  cognitive  overload  in  working 
memory  (Kim  & Pedersen,  2005),  as  well  as  to  generate  criteria  and  attributes  as  much  as  learners  can. 
Particularly,  young  adolescents  do  not  have  enough  working  memory  capacities  to  hold  much  information 
and  use  metacognitive  strategies  at  the  same  time.  One  of  the  design  considerations,  particularly  for 
classification  and  ill-structured  problem-based  learning,  is  structure  that  easily  iterate  the  learning  process. 

• Motivating  with  attractive  visuals  to  motivate  learners:  In  the  process  of  hypothesis  generation  and 
classification,  learners  often  show  that  they  just  conducted  very  limited  numbers  of  trials  to  confirm  their 
activities  (Bruner  et  ah,  1956).  Young  adolescents  need  more  motivation  in  order  to  overcome  this  issue. 

• Exposing  students  to  alternative  criteria  and  attributes  improves  their  hypothesis  generation  process 

• Making  attendance  a meaningful  context  (i.e.  training  courses  to  be  an  Animal  Investigator)  so  that  it 

requires  the  students  to  be  energetically  in  the  activity,  and 

• Making  them  feel  as  young  experts  in  real-world  problem  in  the  Problem  solving  phase 

Most  of  the  above  is  achieved  within  Animal  Investigator  modules,  but  some  of  design  principles  might  be 
reconsidered  to  increase  learner’s  performance  according  to  learner’s  feedback. 
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Development  of  Animal  Investigator 


This  program  was  developed  using  Adobe  Flash  professional  8 and  Photoshop  CS,  which  also,  were  used  to  write 
the  user  data  on  the  server  by  PHP  and  MySql  database.  Instead  of  authoring  tools  for  developing  educational 
software,  Adobe  Flash  was  used  because  it  is  a flexible,  powerful,  and  integrated  development  environment  that 
overcomes  the  limitations  of  the  authoring  tool  in  terms  of  vector  graphics  and  scripting  language  like  JavaScript, 
called  ActionScript.  With  regard  to  the  other  development  tools,  learners  will  generate  lots  of  data  while  they  run  the 
program  because  of  the  iteratively  designed  program  environment  that  supports  learners’  activities.  The  data  created 
by  learners  is  shared  in  the  time  stamps  of  a learner,  his  responses  to  questions,  rules  and  attributes  in  the 
classification  stage,  and  more.  Thus  the  database  system  and  the  server-side  script  employed  effectively  store  these 
learner's  data.  As  we  predicted  for  the  user  data,  twenty-eight  students  generated  870  records  during  sixty  minutes  in 
this  pilot  test.  Each  piece  of  data  has  specific  codes  on  the  records  in  order  to  recognize  the  meaning  and  select 
easily  from  database  system. 


Pilot  Evaluation 

The  Animal  Investigator  was  evaluated  by  sixth  grade  students  in  Korea  to  explored  students’  preferences  about 
various  aspects,  such  as  Interest/Enjoyment,  Ease  of  Use,  and  Epistemological  Awareness.  Thus  we  collected  data  to 
evaluate  the  design  of  Animal  Investigator  from  two  surveys  - Constructivist  Internet-based  Learning  Environment 
Survey  for  Science  (CILES-S)  and  Intrinsic  Motivation  Inventory  - and  students’  comments. 

Constructivist  Internet-based  Learning  Environment  Survey  for  Science  (CILES-S).  In  terms  of  CILES-S,  students' 
preferences  for  the  constructivist  Internet-based  learning  environment  were  measured  to  collect  data,  with  the 
author's  permission.  CILES-S  developed  by  Tsai  (2005)  was  developed  originally  by  Chuang  and  Tsai  (2005)  and 
Wen,  et  al.  (2004).  A detailed  description  of  the  four  scales  which  are  appropriate  for  the  Animal  Investigator 
environment  is  presented  as  the  following:  Ease  of  use  measuring  perceptions  to  the  extent  to  which  students  prefer 
the  environment  and  find  easy  to  use,  reflective  thinking  scales  which  are  measuring  perceptions  to  the  extent  to 
which  students  prefer  to  have  the  opportunities  to  promote  critical  self-reflective  thinking  in  the  environments, 
critical  judgment  scales  assessing  perceptions  to  the  extent  to  which  students  prefer  to  have  opportunities  to 
critically  evaluate  information,  and  epistemological  awareness  scale  assessing  perceptions  to  the  extent  to  which 
students  prefer  to  have  opportunities  to  explore  the  value,  source,  merit  or  nature  of  knowledge  (Tsai,  2005).  It  is 
administered  in  order  to  assess  the  degree  to  which  the  principles  of  constructivism  have  been  implemented  in  a 
program  or  course.  All  of  these  scales  were  designed  to  investigate  students'  preferences  towards  constructivist 
Internet-based  learning  environments.  Original  CILES-S  was  modified  to  the  voice  for  the  sixth  graders  for  this 
study  because  the  questionnaire  was  designed  for  use  with  high  school  students  and  then  translated  into  Korean.  The 
overall  scale  appears  to  be  consistent  internally  with  an  alpha  coefficient  of  .92. 

Student's  mean  scores  on  the  four  scales  of  the  CILES-S  show  the  twenty-eight  students  average  item  scores  and 
standard  deviations  on  the  four  scales  of  the  CILESI-S,  Table  1.  Students  show  high  scores  across  four  scales:  Ease 
of  use  ( M=  3.36),  Reflective  thinking  (M=  3.25)  Critical  judgment  ( M=  3.50),  and  Epistemological  awareness  (M= 
3.31).  Students’  scores  on  the  Reflective  thinking  scale  ( M=  3.26)  were  relatively  lower  when  compared  to  those  of 
other  scales. 


Table  1.  Descriptive  Statistics  for  the  CILES-S  (n  = 28) 


Scale 

No.  of  items 

Item  mean 

S.D. 

Ease  of  Use 

5 

3.36 

.71 

Reflective  Thinking 

5 

3.25 

.87 

Critical  Judgment 

4 

3.50 

.76 

Epistemological  Awareness 

5 

3.31 

.81 

We  also  compared  between  male  students'  (n=ll)  and  female  students'  (n=17)  preferences  for  Animal  Investigator. 
The  results  of  the  pilot  study  do  not  show  significant  differences  between  them  on  the  each  scale.  As  shown  in  Table 
2,  the  results  of  the  present  study  show  that  male  students  show  higher  preferences  on  the  Ease  of  use  scale  but  with 
no  significant  difference.  On  the  other  hand,  the  results  of  the  reflective  thinking,  critical  judgment,  and 
epistemological  awareness  scales  show  that  female  students  show  a slightly  higher  than  male  students. 
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Table  2.  Gender  comparisons  on  the  scales  of  the  CILES-s  (n  = 28) 


Scale 


Gender 


Mean 


S.D. 


t-value 


Ease  of  Use 
Reflective  Thinking 
Critical  Judgment 
Epistemological  Awareness 


Male 

3.55 

.81 

1.10  (n.s.) 

Female 

3.25 

.63 

1.34  (n.s.) 

Male 

3.11 

.69 

-.68  (n.s.) 

Female 

3.34 

.97 

-.63  (n.s.) 

Male 

3.39 

.77 

-.63  (n.s.) 

Female 

3.57 

.77 

-.63  (n.s.) 

Male 

3.27 

.93 

-.22  (n.s.) 

Female 

3.34 

.75 

-.21  (n.s.) 

Note:  n.s.:  not  significant, 
male  n = 1 1,  female  n = 17. 


Intrinsic  Motivation  Inventory  (IMI)  Students  (n  = 13)  responded  to  the  Intrinsic  Motivation  Inventory  (IMI)  in 
order  to  determine  the  satisfaction  of  the  Animal  Investigator.  Intrinsic  interest  and  satisfaction  are  emotions 
assumed  to  characterize  intrinsic  motivation  (Deci  & Ryan,  1985).  Originally,  the  IMI  contained  16  items  of  several 
headings,  including  seven  subscales  of  intrinsic  motivation:  interest/enjoyment,  perceived  competence, 
effort/importance,  pressure/tension,  perceived  choice,  value/usefulness,  and  relatedness.  The  tone  of  the  IMI 
questions  were  modified  to  be  the  suit  the  target  students,  sixth  graders,  and  then  translated  into  Korean.  All 
responses  were  indicated  on  a 7-point  Likert  scale  that  ranges  from  not  at  all  true  (1)  to  very  true  (7).  Learners 
individually  completed  the  web-based  IMI  for  the  Animal  Investigator  environment  after  completing  their  activity. 
To  examine  students’  intrinsic  motivation  for  using  Animal  Investigator,  we  selected  three  scales  from  the  original 
seven:  interest/enjoyment,  perceived  competence,  and  effort/importance.  These  items  on  the  three  scales  are 
considered  relative  and  helpful  is  improving  Animal  Investigator.  Thus,  average  scores  were  calculated  for  the 
questions  in  the  three  subscales.  The  reason  for  fewer  subjects  than  CILES-S  is  the  lack  of  performance  time  using 
the  Animal  Investigator  for  eighty  minutes  (actual  performance  time  for  two  training  was  about  sixty  minutes 
because  of  conducting  CILES-S  and  IMI  surveys).  The  overall  scale  appears  to  be  consistent  internally  with  an 
alpha  coefficient  of  .87. 

To  analyze  students’  intrinsic  motivation  toward  Animal  investigation,  students’  mean  scores  on  three  scales  of  the 
IMI  were  reviewed.  Thirteen  students’  (male  n=8,  female  = 5)  average  item  scores  and  standard  deviations  are  on  the 
four  scales  of  the  IMI,  as  presented  in  Table  3.  Most  of  the  average  scores  on  the  three  scales  show  high  scores 
except  perceived  competence  (M  = 3.54).  Particularly,  an  item  on  the  perceived  competence  scale,  "I  think  I did 
pretty  well  at  Animal  Investigator  activity,  compared  to  other  students,"  shows  a lower  average  score  (mean  = 3.38). 
On  the  other  hand,  in  the  same  scale,  an  item,  "I  was  pretty  skilled  at  this  activity"  shows  a higher  average  score  (M 
= 4.0). 


Table  3.  Descriptive  Statistics  for  the  IMI  (n  = 13) 


Scale 

No.  of  items 

Item  mean 

S.D. 

Interest/Enj  oyment 

7 

3.90 

1.02 

Perceived  Competence 

6 

3.54 

1.23 

Effort/Importance 

5 

4.32 

.95 

To  analyze  the  reason  for  lower  average  scores  on  the  perceived  competence  scale,  we  compared  male  students' 
(n=8)  and  female  students'  (n=5)  motivation  for  playing  Animal  Investigator.  Even  if  the  number  of  the  subject  is 
small,  total  thirteen,  there  are  significant  differences  between  female  and  male  students’  responses  on  the  scale.  As 
presented  in  Table  4,  the  results  of  the  gender  comparisons  shows  that  female  students  are  1.03  point  lower  than 
male  students  on  the  scale. 
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Table  4.  Gender  comparisons  on  the  scales  of  the  IMI  (n  = 13) 

Scale  Gender  Mean  S.D. 


Interest/Enj  oyment 
Perceived  Competence 

Effort/Importance 

Note:  *p<0.05. 
n.s.:  not  significant. 

Male  n = 8,  female  n = 5. 


Male 

4.23 

1.07 

Female 

3.37 

.76 

Male 

3.90 

1.47 

Female 

2.97 

.30 

Male 

4.45 

1.08 

Female 

4.12 

.78 

t-value 
1.563  (n.s.) 

1.375  * 

.588  (n.s.) 


Student  Comments  With  two  surveys,  students  wrote  comments  about  more  detail  comments  for  the  next  version 
of  Animal  Investigator.  The  following  comments  are  summarized  with  regard  to  the  interface  design  and  contents:  1) 
many  students  suggested  that  providing  a more  interactive  main  menu  to  go  back  and  forth  between  stages  would  be 
helpful,  and  2)  some  students  pointed  out  that  the  contents  in  the  classification  made  it  somewhat  difficult  for  them 
to  find  criteria.  In  the  overall  satisfaction,  3)  several  students  are  satisfied  with  the  various  sources,  overall  interface, 
and  activities  in  the  Animal  Investigator,  and  3)  some  of  the  students  commented  that  this  program  will  be  helpful  to 
their  learning. 


Discussions  / Conclusion 

In  this  study,  we  explore  students’  preferences  toward  students’  problem-solving  environments  designed  to  attain  the 
goal:  a tool  to  enhance  learners’  knowledge  generation  using  two  strategies,  classification  and  hypothesis  generation. 
In  terms  of  the  results  of  students’  scores  (M=  3.31,  n = 28)  in  the  Epistemological  beliefs  scale  on  the  CILES-S, 
Animal  Investigator  can  play  a role  as  a learning  environment  to  support  students’  learning  activities  for  knowledge 
construction.  With  regard  to  this  relationship  between  learning  environment  and  knowledge  construction,  Tsai  and 
Chuang  (2005)  have  asserted  that  there  is  an  interplay  between  students’  epistemological  beliefs  and  learning 
preferences  for  learning  environments,  as  well  as  a fundamental  step  for  exploring  learner’s  learning  processes  and 
outcomes  in  the  environment.  However  we  hold  up  evaluating  whether  or  not  Animal  Investigator  influences 
students’  learning  outcomes,  especially  ill-structured  problem-solving  contexts,  future  research  is  planned.  Future 
research  on  the  topic  will  expand  further  our  understanding  of  the  process  of  knowledge  generation,  strategies  to 
support  the  process,  and  the  tool,  and  environments  to  facilitate  the  process. 
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Abstract 

The  purpose  of  this  paper  is  to  present  a model  media  and  animation  program  for  minority  students  to  build 
stronger  connections  between  theory  and  practice.  New  technologies  in  the  animation  field  enable  students  to 
expand  their  educational  goals  as  they  leam  reading,  writing,  math  and  science  around  the  animation  curriculum. 
Currently,  a predominantly  Latino  community  in  California  is  setting  up  a model  program  to  teach  animation  as  an 
at-risk  and  gang-related  intervention  strategy. 

Introduction 

“As  technology  continues  to  be  an  important  part  of  children’s  lives  outside  of  school,  it  is  shaping  their 
expectations  of  what  school  will  be  like.  The  National  Education  Plan  intends  to  explore  this  trend  and  the 
implications  for  creating  digital  age  educational  opportunities  to  match  the  expectations  of  digital  age  students” 
(Paige,  2003).  The  purpose  of  this  paper  is  to  present  a model  animation  program  for  minority  students  in  media  to 
build  stronger  connections  between  theory  and  practice.  New  animation  technologies  enable  students  to  expand  their 
educational  goals  as  they  understand  the  relevance  of  learning  reading,  writing,  math  and  science  around  the 
animation  curriculum. 

Need  for  Program 

Only  68  percent  of  students  entering  high  school  earn  a standard  high  school  diploma.  Another  16  percent 
eventually  receive  an  alternative  diploma,  such  as  a GED.  (McCabe,  2000)  A longitudinal  study  by  the  U.S. 
Department  of  Education  offers  information  about  drop  out  rates:  “African-American  and  Hispanic  students  are  less 
likely  to  be  enrolled  in  a college  preparatory  track  (25.7  % and  22.6%  respectively)  than  either  Asian  (42.1%)  or 
white  (34.1%);  in  2000,  drop  out  students  were  defined  as:  “24%  whites,  21%  Asians,  43%  Native  Americans,  45% 
African-American,  and  47%  Hispanics  students”  (Institute  of  Education  Sciences,  2002).  Other  students  who 
needed  the  most  extensive  remediation  (more  than  two  semesters  of  reading)  were  six  times  less  likely  to  earn  a BA 
than  those  who  required  no  remedial  work  (Adelman,  1999). 

Animation  Curriculum 

The  animation  curriculum  is  a four  to  six  year  course  of  study  that  is  engaging,  interactive,  diversified,  and 
based  on  the  multiple  intelligences  model.  Each  learning  module  is  relevant  and  supported  by  state  standards,  which 
is  easily  adaptable  for  each  state  in  the  union  as  the  virtual  high  school  expands  its  services.  The  bonus  to  the  model 
is  that  the  content  is  designed  for  an  online  delivery  system,  which  increases  access  and  equity  for  minority  students 
and  teachers. 

The  flow  of  the  animation  curriculum  provides  concrete  learning  outcomes  and  skill  sets  at  each  level  of 
the  major  concepts  overview  for  7 -12th  grades.  Art  teachers  serve  as  online  mentors,  and  students  focus  on  real 
world  scenarios  in  the  lesson  modules  as  they  build  a work  portfolio  for  future  employers.  Figure  1 shows  teachers 
learning  how  to  use  the  animation  curriculum  from  drawing  skills  to  computer  technology  integration  at  a private 
workshop.  (Lightfoot,  2006) 


245 


Figure  1.  Teachers  at  an  animation  workshop. 


School-to-Work  Career  Path 

Application  of  animation  models  using  digital  media  will  provide  motivation  for  teachers  to  integrate 
technology  into  cross-disciplinary  modules  matched  with  supplemental  state-  approved  textbooks.  In  order  to 
demonstrate  mastery  of  curricular  standards,  a student  will  be  able  to  demonstrate  critical  thinking  skills  during 
lessons,  engage  in  community  service  activities,  and  present  an  electronic  portfolio  for  graduation  requirements. 
Animation  is  integral  to  leading  areas  of  employment  over  the  next  20  years  in  these  fields: 

1.  Wireless  games,  jokes,  cartoons,  greetings  on  mobile  phones 

2.  Commercials  and  advertisements 

3.  Video  Games 

4.  Entertainment:  high  production  movies 

5.  Educational  software  and  “edutainment” 

6.  Web  page  design 

7.  Architectural  fly-over  and  walk-through 

8.  Court  re-enactment  for  criminal  cases 

9.  Insurance  claims  and  accident  simulations 

10.  Flight  simulators  and  pilot  training 

1 1 . Aircraft  construction  and  planning 

12.  Military  tank  training  and  war  games 

13.  Medical  field-virtual  surgery  for  doctors 

14.  Burn  therapy  using  virtual  distraction  therapy 

15.  Corporate  presentations  and  mergers 

16.  Clothing  design  with  virtual  models 

17.  Pharmaceutical  companies-molecular  structure 

18.  Anthropology-recreation  of  lost  species 

19.  Law  enforcement-forensics 

20.  Bio-med  research  and  development 

21.  Dentistry  and  plastic  surgery  (“A  World  Class  Animation”,  2005). 

Program  Implementation 

A federal  grant  team  at  National  University  developed  an  interest  in  setting  up  animation  labs  as  a 
technology  tool  for  interdisciplinary  project-based  learning  for  Title  I schools  in  California:  1)  Montgomery  Middle 
School  in  Cajon  Valley  School  District;  2)  Corky  McMillin  Charter  School  in  East  Lake  near  Bonita;  3)  National 
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University  Costa  Mesa  Learning  Center,  as  a portable  lab  for  our  NU  graduate  students  and  pre-service  teachers 
enrolled  in  PT3  grant  workshops  and  related  district  programs;  and  4)  at  a JCCS  court  school  in  San  Marcos  for 
incarcerated  youth.  Currently,  a predominantly  Latino  community  in  California  is  setting  up  a model  program  to 
teach  animation  as  an  at-risk  and  gang-related  intervention  strategy.  This  community  has  tried  many  programs  in  the 
past  twenty  years,  but  the  animation  program  is  the  first  one  that  will  capture  the  interest  of  the  students,  increase 
student  retention  and  also  lead  to  a solid  career  path.  During  the  program,  students  will  set  up  a business  model 
relationship  through  Prabble  Animation  Labs  (PAL)  with  their  community  to  create  Public  Service  Announcements 
(PSAs)  and  other  commercial  products  for  web,  television,  film,  & private  ventures.  A PAL  Community  Builder 
Model  will  encourage  business  and  industry  to  employ  these  Latino  students  and  teachers  during  the  media  and 
animation  program  to  create  advertising,  public  service  announcements,  and  websites;  and  to  illustrate  comic  books, 
graphic  novels,  and  computer  video  games  as  shown  in  Figure  2.  (Lightfoot,  2006) 


Figure  2.  Teachers  as  animation  artists  at  work. 


Animation  Industry  Growth 

Many  3-D  animators  from  the  video  game  industry  have  new  jobs  creating  special  effects  in  television 
series  and  films  because  of  their  ability  to  create  scenes  that  cannot  be  filmed  such  as  the  beheading  in  The  Last 
Samurai  movie  (Animation  World  Network,  2006);  the  craving  for  realism;  and  the  issue  of  safety  during  stunts 
(Skweres,  2003).  The  3-D  animation  companies  are  forecasted  for  growth  in  Los  Angeles,  California  as  major 
studios  like  DreamWorks,  Disney,  and  Sony  are  anticipated  to  add  3-D  animation  departments.  The  expansion  of 
major  movie  studios  such  as  Warner  Bros.,  20th  Century  Fox,  Viacom  and  the  new  DreamWorks  SKG  studios  with 
investments  of  $1  billion  in  their  animation  departments  shows  their  commitment  to  this  emerging  field: 

The  best  bet  is  to  have  art  skills  and  a good  knowledge  of  computers.  In  the  artistic  area,  various 
software  programs  are  used  to  create  three-dimensional  backgrounds  for  film,  video  games,  the 
Internet  and  many  other  multimedia  uses.  Computers  are  also  used  to  create  special  effects  and 
graphics,  and  color  or  draw  animation  art.  The  animators  and  special  effects  departments  find  that 
they  constantly  must  outdo  themselves,  as  the  audience  appetite  for  fresh  "eye  candy"  continues 
to  grow. . .Films  are  just  one  part  of  the  industry,  though.  Actually,  most  animation  studios  make 
their  money  through  commercials  rather  than  feature  films.  Animation  and  special  effects 
professionals  can  find  work  in  many  other  forms  of  visual  media  (Saludos  Hispanos,  2000). 
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Professional  seminars  are  a common,  yet  under-studied,  learning  environment.  For  purposes  of  this  study  a 
professional  seminar  is  defined  as  a training  environment  where  invited  experts  present  timely  information  to 
professionals  with  the  intent  that  the  learners  can  use  that  information  in  their  workplace.  This  definition  is  general 
enough  to  include  learning  environments  of  different  names,  such  as  symposia,  workshops,  or  conferences.  Malcolm 
Knowles  (1970)  wrote  of  conferences:  “The  full  potential  of  conferences  ...  as  formats  for  learning  is  far  from  being 
realized,  largely  because  the  conference  planners  don’t  perceive  them  as  formats  for  learning”  (p.  140).  We  share 
this  sentiment,  and  believe  that  applying  systematic  instructional  design  procedures  to  the  planning  and 
implementation  of  seminars  can  help  realize  the  potential  learning  value  of  seminars  for  professionals  in  any 
discipline. 

We  sought  to  answer  a broad  question:  Flow  should  systematic  instructional  design  be  done  in  the  context 
of  a professional  seminar?  Our  goal  was  to  formulate  practical  recommendations  and  guidelines  for  instructional 
designers  and  seminar  planners.  The  context  in  which  this  study  was  carried  out  was  a client’s  pre-existing  seminar, 
the  National  Security  Policy  (NSP)  seminar,  hosted  by  a government  management  and  leadership  training  facility. 
This  paper  will  present  a case  study  of  the  two  phases  that  comprised  this  project,  namely,  a needs  assessment  and 
the  subsequent  seminar  redesign. 

The  methods  used  to  conduct  this  study  combined  the  processes  involved  in  doing  instructional  design  for 
an  existing  instructional  environment  with  systematically  researching  the  development  of  that  project.  As  Richey 
(1997,  p.  93)  explains,  this  combination  of  “doing  development  and  studying  development”  is  what  is  referred  to  as 
developmental  research.  Instructional  designers,  program  planners,  training  directors,  and  any  others  involved  with 
the  planning  of  seminar-style  instructional  events  may  find  the  findings  of  this  study  useful. 

Unique  Characteristics  of  Seminars 

In  order  that  we  ground  our  research  about  design  in  this  one  environment  in  theory  and  practice,  some 
generalization  is  necessary.  Therefore,  we  sought  to  identify,  from  the  literature,  characteristics  common  to  these 
various  learning  environments.  This  review  also  served  to  provide  useful  data  for  our  Phase  1 needs  assessment.  We 
identified  four  unique  characteristics  of  seminars:  (1)  diverse  learn  characteristics,  (2)  focus  on  complex  problem 
solving,  (3)  passive  learning  environment,  (4)  brief  duration  of  instruction.  Each  characteristic  is  briefly  described 
below. 

Seminar  attendees  generally  represent  a variety  of  different  agencies,  organizations,  or  professional  fields. 
This  provides  attendees  with  opportunities  exchange  ideas  with  and  learn  from  others  (Mundry,  Britton,  Raizen  & 
Loucks-Florsley,  2000).  Seminar  content  tends  to  focus  more  on  information  that  can  be  adapted  for  use  in  the 
workplace,  rather  than  training  on  specific  job  skills.  This  makes  seminars  ideal  environments  for  problem  solving 
and  decision  making  activities  (Mundry  et  al.,  2000).  Instead  of  being  active  learning  environments,  unfortunately 
some  seminars  resemble  extended  exercises  in  listening  to  presenter  after  presenter  (Mundry  et  al.,  2000).  Bradford 
and  Corey  (1970)  speak  of  this  in  terms  of  a psychological  “gap  between  the  platform  and  the  audience”  (p.  68). 
Professional  seminars  are  often  one-time,  stand-alone,  or  annual  events.  A presenter  may  have  an  hour,  a morning, 
an  afternoon,  or  maybe  one  full  day  to  present.  This  means  that  instruction  must  be  of  an  appropriate  scope  in  order 
to  not  merely  present  information,  but  also  to  provide  time  for  practice,  feedback,  and  interaction  (Gagne,  Briggs  & 
Wager,  1992)  and  participants  may  need  to  prepare  prior  to  attending  (Boucouvalas,  1985). 

Purpose  of  This  Study 

Given  that  professional  seminars  are  unique  learning  environments,  how  should  systematic  instructional 
design  be  done  in  the  context  of  a professional  seminar?  Recommendations  in  the  literature  provide  answers  to  this 
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question,  but  very  little  empirical  research  exists  to  verify  these  recommendations.  In  this  study  we  tried  to  generate 
empirical  data  to  add  to  the  existing  body  of  research. 

The  purpose  of  this  study  was  to  document  the  use  of  instructional  design  processes  in  the  redesign  and 
implementation  of  a professional  seminar.  The  case  study  resulting  from  this  developmental  research  study  can  be 
divided  into  two  distinct  phases  of  data  collection:  a front-end  needs  assessment  and  the  instructional  redesign, 
implementation,  and  evaluation  of  an  existing  seminar  for  a client.  The  main  research  question  driving  the  study 
was:  How  can  instructional  design  procedures  be  used  in  the  planning  and  production  of  a face-to-face  professional 
seminar?  As  part  of  this  main  question,  we  had  three  working  research  questions.  (1)  What  attitudes  and 
expectations  about  learning  at  seminars  do  professionals  hold?  (2)  What  are  the  optimal  instructional  delivery 
methods  for  a seminar  setting?  (3)  How  can  seminar  attendees’  learning  be  assessed? 

This  paper  is  divided  into  two  main  sections.  The  first  section  includes  the  methods,  results,  and  a 
discussion  of  those  results  of  Phase  1 of  the  study,  a needs  assessment.  The  second  section  includes  the  methods, 
results  and  a discussion  of  those  results  of  Phase  2 of  the  study,  a redesign  of  the  client’s  National  Security  Policy 
(NSP)  seminar.  Overall  conclusions  of  the  study  are  also  provided. 

Phase  1:  Needs  Assessment 
Methods  Used  in  the  Needs  Assessment 

A needs  assessment  was  conducted  using  methods  suggested  by  Rossett  (1987).  These  methods  included 

(1)  surveying  a sample  of  the  client’s  target  audience  regarding  their  attitudes  towards  and  expectations  of  seminars, 

(2)  reviewing  the  literature  for  recommendations  on  how  seminars  can  best  be  designed,  and  (3)  documenting  gaps 
between  recommendations  for  designing  seminars  and  the  client’s  actual  procedures  for  doing  so.  The  needs 
assessment  would  lead  us  to  identify  optimal  instructional  delivery  methods  to  consider  during  Phase  2 of  the  study. 

Survey  of  Target  Audience  Attitudes 

A needs  assessment  survey  instrument  was  constructed,  field  tested,  and  revised  to  identify  attendees’ 
attitudes  and  expectations  regarding  seminars,  how  relevant  they  perceive  seminars  to  be,  and  factors  that  they  feel 
inhibit  and  facilitate  their  learning  at  seminars.  The  instrument  consisted  of  15  items,  13  closed-ended  items  and  2 
open-ended  items.  The  instrument  was  delivered  via  a web-based  survey  site.  The  client  provided  a database  of  e- 
mail  addresses  of  attendees  at  their  seminars,  including  past  NSP  seminars.  An  e-mail  invitation  to  complete  the 
questionnaire  containing  a hyperlink  to  the  questionnaire  was  sent  to  approximately  2,100  individuals.  The  survey 
remained  open  and  active  on  the  site  during  approximately  four  weeks,  during  which  262  complete  questionnaires 
were  submitted. 

Seventy  percent  of  respondents  were  male  and  thirty  percent  were  female.  Forty-three  percent  of  the 
respondents  identified  themselves  as  Department  of  Defense  (DOD)  employees  and  another  forty-eight  percent 
identified  themselves  as  employees  of  various  other  federal  agencies.  The  remaining  9%  of  the  respondents  worked 
for  either  state  or  local  governments  or  private  organizations.  The  client  verified  that  this  demographic  breakdown  is 
representative  of  the  target  audience  of  its  seminars. 

Literature  Search 

We  conducted  a search  of  online  databases  containing  research  in  fields  including  instructional  design  to 
identify  research  conducted  in  or  about  seminars,  including  Education  Abstracts  and  EBSCO.  Our  goal  was  to 
identify  recommended  instructional  design  processes  for  designing  seminars.  This  literature  review  served  to 
provide  theoretical  background  to  the  study  and  also  as  a data  source  for  our  needs  assessment.  Many  of  the  results 
of  this  initial  search  offered  case  studies  or  narrative  accounts  of  seminars,  most  of  which  were  in  higher-education 
settings.  Therefore,  the  search  was  widened  to  include  non  research-based  literature  and  books.  This  second  search 
yielded  approximately  10  articles  and  books  addressing  the  design  and  implementation  of  seminar-style  events,  and 
provided  some  data  regarding  best  practices.  Some  of  the  sources  directed  their  guidelines  to  the  design  of 
conferences  or  meetings,  but  were  judged  appropriate  for  this  study. 
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Review  of  the  Client’s  Design  Processes 

We  held  bi-weekly  phone,  email,  and  face-to-face  discussions  with  the  NSP  Program  Director  to  discuss 
how  he  and  his  staff  plan  and  implement  the  seminar.  We  used  this  data  to  compare  against  the  recommended 
processes  found  in  the  literature  and  identify  how  the  client  might  improve  its  seminar  design  processes. 

The  client  produces  and  hosts  numerous  different  seminar  programs  for  federal  supervisors,  managers,  and 
executives  each  year.  The  NSP  seminar  belongs  to  a series  of  seminars  focused  on  policy  issues,  as  has  been 
conducted  annually  at  the  center  for  approximately  30  years.  The  seminar  generally  lasts  for  nine  days,  each  day 
devoted  to  a different  topic  and  featuring  a single  presenter.  A presenter  speaks  for  three  to  four  hours  in  the 
morning  before  breaking  for  lunch,  after  which  the  presentation  continues.  Presenters  have  discretion  over  the  use  of 
instructional  materials  and  the  types  of  interaction  they  employ.  Their  presentations  are  rarely  critiqued  or  revised  by 
the  client.  Presenters  are  invited  subject-matter  experts  in  a field  related  to  theme  of  the  seminar,  frequently 
academics,  retired  government  or  military  personnel. 

We  compiled  a list  of  common  instructional  design  guidelines  based  on  several  sources  (Allen,  1996; 
Branch,  1994;  Dick,  Carey  and  Carey,  2001;  Martin,  1990;  Smith  & Ragan,  1999;  Wedman  & Tessmer,  1993)  with 
which  we  could  compare  the  processes  that  the  client  used  with  commonly  accepted  ID  processes.  The  list  was 
referenced  as  we  reflected  on  our  discussions  with  the  client,  but  was  not  shared  with  the  client.  Table  1 displays  the 
list  of  common  instructional  design  guidelines  we  compiled. 

Table  1 

Common  Instructional  Design  Guidelines 


Conduct  a needs  assessment  Develop  instructional  materials 

Analyze  learning  environment  Conduct  formative  evaluation  of  materials 

Conduct  learner  analysis  Revise  instructional  materials 

Define  learning  goals  and  objectives  Evaluate  outcomes  of  instruction 

Develop  evaluation  instillments  Conduct  summative  evaluation  of  instruction 

Design  delivery  strategies 


Results  of  the  Needs  Assessment 


Survey  of  Target  Audience  Attitudes 

Table  2 displays  frequency  responses  to  two  different  items  on  the  needs  assessment  survey.  The  first  item 
asked  respondents  to  indicate  their  agreement  with  three  statements:  “I  enjoy  the  seminars  I attend,”  “The  seminars  I 
attend  benefit  me  professionally,”  and  “I  create  benefits  for  my  organization  by  attending  seminars.”  The  second 
item  asked  “How  would  you  rate  the  importance  of  the  following  factors  for  helping  you  learn  while  at  a seminar?” 

Table  2 


General  Attitudes  Towards  Seminars 


Statement 

SA 

A 

SA/SD 

D 

SD 

I enjoy  the  seminars  I attend. 

98 

146 

32 

1 

0 

The  seminars  I attend  benefit  me  professionally. 

95 

147 

34 

1 

0 

I create  benefits  for  my  organization  by  attending  seminars. 

24 

72 

89 

63 

29 

Importance  of  Factors  That  May  Influence  Learning  at  Seminars 
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Factor 

VH 

H 

M 

L 

VL 

Covering  topics  relevant  to  my  job  or  interests 

164 

86 

18 

2 

1 

Having  a variety  of  session  formats,  more  than  just  lectures 

105 

114 

38 

8 

2 

Discussing  topics  thoroughly  with  attendees  and  presenters 

91 

122 

49 

9 

0 

Being  able  to  attend  every  session  I want  to  attend 

77 

128 

54 

10 

1 

Actively  participating  and  interacting,  more  than  just  listening 

103 

100 

51 

11 

2 

Having  additional  resources  about  topics,  beyond  just  the 
presenter  slides 

84 

115 

56 

11 

1 

Having  ample  informal  socializing  or  networking  opportunities 

61 

94 

80 

31 

4 

Note : SA=Strongly  Agree,  A=Agree,  SA/SD=Somewhat  Agree/Somewhat  Disagree,  D=Disagree,  SD=Strongly 
Disagree.  VH=Very  High,  H=High,  M=Medium,  L=Low,  VL=Very  Low. 


Respondents  seemed  to  enjoy  and  see  professional  benefit  in  seminar  attendance.  However,  their  responses 
are  more  varied  regarding  the  benefits  that  seminar  attendance  creates  for  their  organizations.  In  response  to  the 
second  item,  respondents  seemed  to  feel  that  several  factors  were  important  for  helping  them  learn  at  a seminar. 
Chief  among  these  was  the  relevance  of  the  topics  to  their  job  or  interests.  However,  having  a variety  of  session 
formats,  thorough  discussion,  and  active  participation  also  ranked  high.  These  four  factors  are  noteworthy  because 
each  is  mentioned  in  the  literature  as  being  important  for  seminar  design. 

Table  3 displays  the  results  from  two  additional  closed-ended  items  on  the  questionnaire.  The  first  was 
“Why  do  you  attend,  or  have  you  attended,  seminars?’  Seven  options  were  then  given  and  respondents  could  select 
all  options  that  applied  to  them.  The  second  item  was  “From  your  experience,  which  of  the  following  things  keep 
you  from  feeling  like  you  have  learned  all  you  could  at  a seminar?”  9 options  were  provided  and  respondents  could 
select  all  options  that  applied  to  them. 

Table  3 

Reasons  for  Attending  Seminars 


Reason 

n 

% 

To  stay  current  with  the  most  up-to-date  issues  in  my  field 

216 

78 

To  be  exposed  to  a broader,  more  general  range  of  issues  related  to  my  field 

212 

77 

To  engage  in  discussions  about  issues  and  solutions  with  others 

202 

73 

To  analyze  and  improve  upon  current  practice  in  my  field 

179 

65 

To  acquire  a skill  I will  use  in  my  job 

163 

59 

To  network 

161 

58 

Other  reason(s) 

32 

12 

Factors  That  May  Inhibit  Learning  at  Seminars 

n 

% 

Too  many  presentation  topics  being  irrelevant  or  inapplicable  to  my  job  or  interests 

144 

53 

Too  much  sitting  and  listening  and  not  enough  participation 

128 

47 

Not  being  able  to  attend  sessions  I thought  would  be  interesting  or  applicable 

127 

47 

Too  few  opportunities  to  discuss  topics  thoroughly  with  attendees  and  presenters 

118 

44 

Too  few  resources  or  materials  to  take  away  or  access 

78 

29 

Not  enough  variety  in  the  format  of  the  sessions 

69 

25 

Too  few  informal  socializing  or  networking  opportunities 

39 

14 

Other  factor(s),  please  specify 

36 

13 

I always  feel  like  I have  learned  all  I could 

27 

10 

Note:  Total  n=271-276 


252 


Table  3 displays  the  response  frequencies  for  each  option  in  the  two  items  described  previously.  More  than 
three-quarters  of  respondents  indicated  that  one  of  the  reasons  they  attend  seminars  is  to  stay  current  with  or  to  be 
exposed  to  issues  in  or  related  to  their  field.  These  were  the  most  frequently  cited  responses.  The  next  most  frequent 
response  was  to  engage  in  discussions  about  issues  and  solutions.  Networking  and  acquiring  job  skills  were  less- 
frequently  cited  responses. 

The  second  item  in  Table  3 deals  with  factors  that  may  inhibit  learning  at  seminars.  The  most  frequently 
cited  factor  dealt  with  the  irrelevance  of  the  topics  (n=  1 44).  The  second  most  frequently  cited  factor  was  insufficient 
participation  («=128).  Also  of  note  is  that  having  too  few  opportunities  to  discussion  topics  thoroughly  was  the 
fourth  most  frequently  cited  factor  that  may  inhibit  learning  («=1 18). 

Literature  Search 

In  the  literature  we  identified  several  optimal  practices  in  designing  instruction  for  professional  seminars. 
Mundry  et  al.  (2000)  list  several  suggested  practices  for  designing  and  implementing  effective  seminar-style  events, 
including  “Establish  and  share  clear  outcomes”  (p.  2).  Briziarelli  and  Tice  (1996),  Benne  and  Demorest  (1970),  and 
Bradford  and  Corey  (1970)  also  speak  of  the  importance  of  communicating  learning  objectives  to  seminar  attendees. 
These  and  other  authors  also  suggest  strategies  such  as  including  potential  attendees  in  the  planning  and  design  of 
the  seminar  (Ilsley,  1985;  Benne  & Demorest,)  and  using  subject-matter  experts  to  facilitate  discussions  (Benne  & 
Demorest;  Mundry  et  al.). 

Several  sources  also  speak  to  the  importance  of  engaging  all  attendees  in  problem-solving  discussions  and 
activities  (Mundry  et  al.,  2000;  Bradford  & Corey,  1970;  Benne  & Demorest,  1970).  Several  interactive  instructional 
delivery  methods  have  been  shown  to  produce  beneficial  outcomes,  and  may  be  appropriate  for  these  kinds  of 
activities.  These  include  small  group  learning  (Springer,  Stanne  & Donovan,  1999;  Flynn  & Klein,  2001), 
collaborative  learning  (Uribe,  Klein  & Sullivan,  2003),  cooperative  learning  (Johnson,  Johnson,  Stanne  & Garibaldi, 
1990;  Qin,  Johnson  & Johnson,  1995;  Susman,  1998),  and  problem-based  learning  (Dochy,  Segers,  Van  den 
Bossche  & Gijbels,  2003).  While  none  of  these  methods  appear  to  have  been  examined  in  a seminar  environment,  it 
is  reasonable  to  hypothesize  that  similar  results  could  be  achieved. 

Review  of  Client’s  Design  Process 

From  our  conversations  with  the  client,  we  identified  incomplete  use  of  several  instructional  design 
processes  during  the  planning  and  implementation  of  the  client’s  seminars.  Examples  of  incomplete  processes  used 
by  the  client  include  learner  analysis,  designing  delivery  strategies,  evaluating  instructional  materials  and  evaluating 
the  outcomes  of  the  instruction. 

The  NSP  seminar  program  director  usually  began  the  planning  process  by  identifying  a general  theme  and 
selecting  more  specific  topics  for  the  presenters.  A learner  analysis  was  not  conducted  beyond  the  collection  of 
demographic  information  on  each  registered  attendee.  However,  this  information  was  not  used  to  inform  later 
processes  such  as  selecting  or  sequencing  content,  the  designing  delivery  strategies,  or  for  evaluation. 

Each  of  the  client’s  seminars  identified  “Key  Results”  resembling  instructional  objectives,  although  they 
sometimes  resembled  statements  of  activity  or  were  not  well  written  (Sullivan  & Higgins,  1983).  Formal  content 
analyses  were  not  conducted;  however,  the  client  did  regularly  update  the  topics  of  a seminar  to  reflect  current  issues 
or  trends.  For  example,  since  9/11,  the  NSP  Seminar  has  had  a greater  emphasis  on  terrorism  and  terrorist  groups 
that  threaten  U.S.  national  security.  It  was  not  clear  to  us  how  decisions  about  delivery  strategies  were  made  based 
on  the  Key  Results.  In  most  instances,  instructional  delivery  resembled  a lecture,  with  some  interaction  and 
discussion.  Variations  to  this  standard  were  the  result  of  a presenter’s  planning,  not  the  client’s.  A formative 
evaluation  of  the  presenters’  instruction  did  not  occur. 

Evaluations  of  seminar  outcomes  were  conducted  by  an  end-of-course  evaluation  sheet.  Formal  measures 
of  learning  were  not  used,  but  the  client  did  conduct  seminars  that  included  activities  requiring  attendees  to  write  or 
discuss  ideas  and  workplace  application  goals.  Again,  in  many  cases,  if  there  was  an  interactive  or  application- 
oriented  activity  it  was  because  an  individual  presenter,  not  the  client,  initiated  the  activity.  The  outcomes  of  these 
activities,  however,  were  generally  used  for  personal  attendee  reflection  and  not  to  establish  a measure  of  learning. 

Discussion  of  the  Needs  Assessment 

The  results  of  the  needs  assessment  suggested  four  tasks  that  could  be  used  to  improve  the  client’s  current 
planning  and  design  processes.  These  four  tasks  would  be  used  in  the  redesign  of  the  client’s  National  Security 
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Policy  seminar.  These  four  tasks  do  not  represent  a comprehensive  use  of  instructional  design  in  the  planning  and 
implementation  of  a seminar;  that  was  not  the  goal  of  this  study,  and  nor  would  it  have  been  practical. 

The  four  tasks  are  (1)  conduct  a learner  analysis  to  understand  attendees’  prior  knowledge  and  select 
relevant  topics,  (2)  define  and  communicate  instructional  objectives,  (3)  determine  interactive  instructional 
strategies  that  maximize  attendee  participation,  and  (4)  employ  measures  to  evaluate  attendee  learning. 

The  brief  rationale  behind  each  of  these  four  tasks  is  explained  below,  together  with  an  explanation  of  how  they 
were  to  be  conducted  in  the  second  phase  of  the  study. 

A learner  analysis  may  help  indicate  the  relevance  of  topics  to  potential  attendees.  Relevance  refers  to 
attendees’  perception  of  how  the  seminar  will  contribute  value  to  their  professional  practice  and  to  their 
organization,  the  perception  of  the  credibility  of  the  information  presented,  and  their  expectation  of  the  utility  of  the 
information  in  their  job  (Huang  2006;  Richey  1998;  Keller  1987a;  Keller  1987b).  Determining  the  relevance  of 
seminar  topics  could  be  accomplished  by  including  representatives  of  the  target  audience  in  the  planning  of  the 
seminar  or  by  surveying  a sample  of  the  target  audience. 

Systematic  instructional  design  requires  that  objectives  be  specified  during  the  design  of  the  instruction. 
However,  when  individual  presenters  select  content  and  delivery  methods,  defining  and  adhering  to  pre-determined 
objectives  may  seem  more  difficult.  If  the  presenters  understand  the  intended  outcomes  of  the  seminar,  they  can  be 
guided  to  select  the  content  and  delivery  methods  that  will  meet  those  outcomes.  This  may  require  more  oversight 
on  the  part  of  seminar  planners  to  formatively  evaluate  the  presenters’  development  work. 

Responses  to  the  survey  data  indicated  that  seminar  attendees  expect  to  interact  and  participate  actively  in 
seminar  sessions.  Instead  of  relying  exclusively  on  a lecture-based  delivery  format,  seminar  planners  should 
incorporate  more  interactive  delivery  methods  into  the  seminar  agenda.  Doing  so  would  likely  require  careful 
coordination  between  the  planners  and  the  presenters  in  order  to  prepare  necessary  materials  and  train  presenters  in 
the  chosen  delivery  method  techniques,  if  necessary. 

Seminars,  as  defined  in  this  study,  do  not  focus  on  specific  job  skills.  Thus,  measuring  learning  gains  with 
tests  may  be  less  appropriate.  However,  measuring  learning  is  an  important  component  of  systematic  instructional 
design.  Additionally,  measures  of  learning  could  be  a valuable  too  to  justify  the  value  of  seminar  training  to 
employers.  Some  group-based  instructional  activities  such  as  cooperative  learning  or  problem-based  learning  could 
lead  to  groups  or  individuals  producing  a learning  product  (a  case  analysis,  for  example)  that  could  be  submitted  and 
scored.  At  the  same  time,  the  learning  product  could  be  an  acceptable  indicator  of  learning  of  problem-solving 
ability. 


Phase  2:  Seminar  Redesign 

The  purpose  of  Phase  2 of  this  study  was  to  apply  the  conclusions  of  our  needs  assessment  to  the  redesign 
of  a professional  seminar.  In  this  section  of  the  paper,  we  describe  the  methods  and  the  results  of  the  redesign  and 
implementation  of  the  National  Security  Policy  seminar  using  the  four  tasks  derived  from  Phase  1. 

Methods  Used  in  the  Seminar  Redesign 

The  methods  used  during  Phase  2 of  the  study  included  (1)  conducting  a learner  analysis  that  emphasized 
the  relevancy  of  the  seminar  topics,  (2)  redesigning  the  seminar  agenda  to  include  more  interactive  delivery 
methods,  (3)  observing  the  redesigned  seminar,  and  (4)  evaluating  learning  and  attitudinal  outcomes. 

Learner  Analysis 

Since  the  seminar  topics  had  already  been  selected  prior  to  this  study,  we  could  not  include  representatives 
of  the  target  audience  in  the  selection  of  relevant  topics.  Instead,  we  administered  a brief  pre-seminar  attitude 
questionnaire  to  the  registrants  of  the  seminar  to  gauge  the  relevance  of  the  chosen  topics  and  the  registrants’ 
expectations  of  the  seminar.  The  questionnaire  was  comprised  of  seven  items,  six  closed-ended  and  one  open-ended. 

The  questionnaire  was  delivered  via  a web-based  survey  site.  The  client  provided  the  email  addresses  of  the 
registered  participants,  and  an  email  invitation  containing  a hyperlink  to  the  instrument  was  sent  to  each  of  the  21 
registrants.  About  half  of  the  registrants  completed  the  pre-seminar  questionnaire  (n=  12)  in  part  due  to  some 
attendees  registering  for  the  seminar  at  the  last  minute. 
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Redesign  of  the  Seminar  Agenda 

We  collaborated  with  the  client  to  modify  the  general  format  of  the  National  Security  Policy  seminar  from 
a format  dominated  by  lecture-style  presentations  to  one  that  included  more  interactive  delivery  methods.  We 
communicated  regularly  with  the  client  via  phone,  email,  and  face-to-face  meetings  to  propose  variations  of 
interactive  sessions  and  discussed  how  the  sessions  would  be  conducted. 

The  outcomes  of  this  collaboration  were  two  instructional  objectives  and  an  interactive  delivery  strategy 
that  could  help  assess  the  new  objectives.  Since  our  literature  search  indicated  that  seminars  can  be  excellent  venues 
for  problem  solving,  the  two  objectives  were  based  on  problem-solving  outcomes:  “Learners  will  define  the  nature 
of  a current  problem  in  counterterrorism”  and  “Learners  will  develop  a solution  for  a current  problem  in 
counterterrorism.” 

The  statement  of  the  new  instructional  objectives  helped  guide  the  development  of  an  interactive  delivery 
strategy.  A discussion-based  strategy  was  selected  to  create  more  interaction  in  the  NSP  sessions.  This  strategy 
consisted  of  two  formats  of  group  discussion:  small-  and  full-group  discussions.  Small  groups  would  be  comprised 
of  five  to  seven  individuals.  The  focus  of  both  the  small-  and  the  full-group  discussions  would  be  open-ended 
discussion  questions  related  to  the  lecture  presentations. 

Observations  of  Seminar  Sessions 

The  primary  researcher  observed  all  the  seminar  activities  during  the  first  four  days  of  the  seminar.  During 
this  time,  four  different  presenters  spoke  on  four  different  topics:  China;  Ethics  and  National  Security;  India, 
Pakistan  and  South  Asia;  and  Economics  and  National  Security.  A total  of  19  people  attended  the  seminar. 

A timed-interval  classroom  observation  protocol  was  developed  to  record,  at  5-minute  intervals,  several 
related  variables  that  describe  the  classroom  interaction.  The  observation  protocol  was  based  on  a version  used  by 
Atkinson  (2004),  which  was  originally  developed  from  WestEd  (n.d.)  & Horizon  Research  Inc  (2005).  Using  the 
protocol,  an  observer  can  record  how  five  different  variables  are  implemented  in  a classroom:  “Type  of  Instruction,” 
“Student  Engagement,”  “Problem  Solving  Activities,”  and  “Problem  Solving  Facilitation  Techniques.”  The  data 
recorded  on  the  protocol  would  allow  us  to  paint  a detailed  moment-by-moment  picture  of  the  time  spent  on  various 
instructional  activities,  presenters’  delivery  strategies,  and  discussion  group  processes. 

Post-seminar  Attitude  Questionnaire 

At  the  conclusion  of  the  four  days  observed  by  the  primary  researcher,  the  seminar  attendees  were  asked  to 
complete  a brief  questionnaire  about  their  experience  at  the  seminar.  The  purpose  of  the  post-seminar  questionnaire 
was  to  determine  the  attendees’  perception  of  the  effectiveness  of  the  various  delivery  methods,  in  particular,  the 
two  new  discussion  formats  that  had  been  implemented.  We  also  sought  to  determine  the  attendees’  post-seminar 
perception  of  relevancy  of  the  topics.  The  questionnaire  instrument  consisted  of  eight  items,  three  of  which  were 
open-ended  and  five  of  which  were  closed-ended.  The  questionnaire  was  delivered  on  paper.  Attendees  completed 
the  questionnaire  at  the  end  of  the  fourth  day  observed  by  the  primary  researcher. 

Results  of  the  Seminar  Redesign 


Learner  Analysis 

One  question  on  the  re-seminar  attitude  questionnaire  asked  “Why  did  you  decide  to  attend  the  National 
Security  Policy  Seminar?”  Seven  options  were  provided,  and  respondents  could  select  all  the  options  that  applied  to 
them.  The  two  most  frequent  responses  were  “To  stay  current  with  the  most  up-to-date  issues  in  my  field”  («=10) 
and  “To  be  exposed  to  a broader,  more  general  range  of  issues  outside  my  field”  (n— 9).  A similar  question  was 
asked  on  the  Needs  Assessment  Questionnaire  about  why  respondents  attend  seminars  in  general,  and  these  same 
two  responses  were  the  most  frequent  responses. 

Table  4 displays  results  for  two  questions  regarding  attendees’  expectations  of  the  seminar  and  the  expected 
relevance  of  the  seminar  topics.  The  first  of  these  items  asked  respondents  to  indicate  their  level  of  agreement  with 
the  four  given  statements.  The  second  of  these  items  asked  respondents  to  indicate  how  relevant  they  anticipated 
each  of  the  topics  being  to  their  job. 
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Table  4 


Registrants  ’ Expectations  for  the  NSP  Seminar 


Statement 

SA 

A 

SA/SD 

D 

SD 

I anticipate  the  knowledge  and  skills  I gain  at  the  NSP  seminar 

Q 

1 

0 

0 

being  applicable  to  my  job. 

o 

J 

I expect  to  be  given  the  opportunity  to  actively  participate  in  the 

7 

5 

0 

0 

0 

seminar  sessions. 

Given  the  opportunity,  I am  likely  to  actively  participate  in  the 

7 

5 

0 

0 

0 

seminar  sessions. 

If  other  attendees  like  myself  actively  participate  in  the  seminar 

8 

4 

0 

0 

0 

sessions,  I stand  to  learn  from  their  insights  and  experience. 

o 

Registrants  ’ Expectations  for  the  Relevance  of  Seminar  Topics 

Topic 

VH 

H 

M 

L 

VL 

China 

2 

2 

3 

1 

2 

Ethics  and  National  Security 

4 

6 

2 

0 

0 

Indian,  Pakistan  and  South  Asia 

2 

4 

5 

0 

0 

Economics  and  National  Security 

4 

5 

2 

0 

0 

Note : SA=Strongly  Agree,  A=Agree,  SA/SD=Somewhat  Agree/Somewhat  Disagree,  D=Disagree,  SD=Strongly 
Disagree.  VH=Very  High,  H=High,  M=Medium,  L=Low,  VL=Very  Low. 


The  data  in  Table  4 suggest  that  registrants  anticipated  that  the  seminar,  overall,  would  provide 
opportunities  for  active  participation.  The  questionnaire  data  also  suggested  that  the  registrants  felt  that  the  seminar 
would  be  applicable  to  their  jobs  and  would  expand  their  knowledge  of  issues  related  to  their  field.  Their  opinions 
regarding  the  relevance  of  particular  topics  varied,  however. 

Redesign  of  the  Seminar  Agenda 

In  the  revised  agenda,  presentations  were  limited  to  the  morning,  while  afternoons  were  dedicated  to  the 
two  discussion  formats.  The  presenters  each  wrote  three  to  four  problem-solving  discussion  questions  (12  in  all)  that 
were  related  to  their  morning  presentation.  The  small  groups  discussed  one  or  two  questions  at  a time,  for 
approximately  30  minutes,  and,  afterwards,  attendees  typed  their  individual  responses  to  the  questions.  The  groups 
were  not  required  to  reach  a consensus  on  their  answers. 

Each  attendee  was  provided  with  a USB  thumb  drive  on  which  was  loaded  a simple  program  that  displayed 
the  discussion  questions  for  the  topic  and  an  area  to  type  responses.  A sample  screen-shot  of  the  program,  showing  a 
discussion  question  and  one  attendee’s  typed  response  to  the  question,  is  seen  below  in  Figure  1. 
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Figure  1 


The  program  automatically  created  a text  file  containing  the  question  response.  The  thumb  drives  were 
collected  after  each  discussion  session.  The  responses  were  downloaded  to  a file  and  scored  by  the  primary 
researcher.  The  results  of  the  scoring  of  the  discussion  questions  are  not  provided  because  the  authors  feel  the  scores 
are  unreliable  measures  of  the  attendees’  learning  at  the  seminar.  Two  factors  contributed  to  this.  First,  we  had 
requested  that  presenters  prepare  the  questions  prior  to  the  seminar  so  that  we  could  evaluate  and  revise  them  and 
ensure  that  they  conformed  to  an  ill-structured  problem-solving  model  (Jonassen,  1997;  Ge  & Land,  2004). 
Flowever,  none  of  the  presenters  provided  the  questions  prior  to  the  seminar,  and  most  had  not  written  them  at  all 
prior  to  arriving.  Second,  we  had  requested  that  the  presenters  serve  as  a second  scorer  of  the  responses,  in  addition 
to  the  lead  researcher.  Flowever,  only  one  presenter  scored  the  question  responses. 

The  full  seminar  group  reconvened  after  individuals  typed  their  question  responses.  The  presenter 
facilitated  the  full-group  discussion  in  which  individuals  were  asked  to  share  the  results  of  their  discussions  and  also 
asked  questions  of  the  presenter. 

Observations  of  Seminar  Instruction 

Over  the  course  of  the  four  days  observed,  just  over  75%  of  the  seminar  was  taken  up  in  instructional 
activities  (22.3  hours  total,  or  about  5.5  hours  per  day).  Instructional  activities  included  presentations  (34.2%,  10.1 
hours),  full-group  discussions  (20.8%,  6.2  hours),  small-group  discussions  (16.4%,  4.8  hours),  and  individual  work 
time  spent  completing  the  discussion  questions  (3.9%,  1.2  hours).  The  small-  and  full-group  discussions  occupied 
slightly  more  than  40%  of  the  total  instructional  time  of  the  seminar  (approximately  six  hours). 

The  four  presenters  varied  in  the  delivery  techniques  they  used  in  their  presentations.  One  of  the  four 
exhibited  what  is  typical  of  a lecture  style.  There  were  very  few  questions  asked  or  comments  made.  A second 
presenter,  in  contrast,  gladly  abandoned  his  prepared  presentation  after  only  30  minutes  in  order  to  respond  to  the 
many  comments  and  questions  being  posed.  The  remaining  two  presenters  exhibited  a blend  of  lecture  and 
interactive  styles.  One  presenter  commented  to  the  primary  researcher,  without  being  prompted:  “This  is  by  far  the 
best  discussion  I’ve  had  with  a group  in  the  30  years  I’ve  been  coming  to  this  program.” 

We  observed  three  trends  in  the  small  discussion  groups.  First,  on-task  time  spent  discussing  the  questions 
appeared  to  decrease  as  the  seminar  progressed.  Instead  of  discussing  the  questions  in  depth,  some  groups  would 
superficially  answer  the  questions  and  break  to  type  individual  responses.  Second,  more  attendees  participated  in  the 
small-group  discussions  than  in  the  full-group  discussions,  meaning  they  voluntarily  made  comments  or  asked 
questions.  On  average,  about  50%  of  the  attendees  participated  in  the  full-group  discussions  while  we  observed  80  - 
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100%  of  the  attendees  voluntarily  participating  in  their  small  group  discussions.  Third,  some  groups  appeared  to 
have  difficulty  synthesizing  information  from  the  presentation  in  order  to  answer  the  discussion  questions.  Group 
members  frequently  posed  questions  to  each  other  that  nobody  else  in  the  group  knew  how  to  answer. 

Post-seminar  Attitude  Questionnaire 

Table  5 displays  respondents’  post-seminar  perceptions  of  the  relevance  of  the  topics  and  the  benefit  of  the 
discussion-based  delivery  strategy. 

Table  5 


Attendees  ’ Perception  of  Seminar  Topics  and  Discussions 


Statement 

SA 

A 

SA/SD 

D 

SD 

Overall,  the  topics  were  relevant  to  my  interests  or  job. 

9 

2 

4 

0 

0 

The  topics  presented  met  my  expectations  for  the  seminar. 

7 

4 

3 

1 

0 

The  discussion  questions  were  helpful  for  analyzing  the  topics. 

1 

7 

7 

0 

0 

Combining  small-group  and  full-group  discussions  was  helpful 

4 

4 

6 

0 

0 

for  analyzing  topics. 

Overall,  the  small-group  discussions  were  helpful  for 

1 

f. 

A 

1 

0 

understanding  the  topics. 

VJ 

VJ 

Note : SA=Strongly  Agree,  A=Agree,  SA/SD=Somewhat  Agree/Somewhat  Disagree,  D=Disagree,  SD=Strongly 
Disagree. 


In  general,  it  appears  that  attendees  at  the  NSP  seminar  felt  that  the  topics  were  relevant  and  met  their 
expectations.  However,  they  appear  to  have  mixed  reactions  to  the  discussion  strategy.  Although  most  respondents 
seem  to  have  had  positive  views  of  this  format,  many  also  thought  it  was  only  somewhat  helpful. 

The  following  comments  may  help  understand  the  data  regarding  the  discussion  strategy  in  Table  5. 
Respondents  were  asked  “What  was  your  impression  of  the  small-group  discussions  as  a means  to  discuss  and 
analyze  the  topics?  The  first  comment  suggests  that  the  morning  presentations  were  insufficient  to  prepare  attendees 
to  discuss  and  analyze  topics.  “The  limited  data  presented  and  the  broad  backgrounds  of  the  partipants  (sic)  left 
many  of  the  breakout  groups  limited  to  a discussion  between  two  or  three  partipants  (sic)  with  the  others  simply 
watching.”  Other  comments  contradict  this  feeling,  mentioning  that  it  was  helpful  to  have  diverse  viewpoints  on 
topics.  A second  response  to  this  question  suggests  that  the  purpose  of  the  small-group  discussions  may  not  have 
been  made  clear  enough  to  all  attendees.  “I  didn’t  get  it — basically  a rehash  of  the  presentations.  Not  sure  of  the 
learning  objective.” 


Discussion  of  the  Seminar  Redesign 

We  attempted  to  measure  the  perceived  relevance  of  the  selected  seminar  topics  to  the  attendees.  Prior  to 
the  seminar  they  seemed  to  feel  that  the  topics  would  be  relevant  to  them,  and  felt  similarly  after  the  seminar. 
However,  because  the  topics  had  been  selected  prior  to  this  study,  this  outcome  cannot  be  attributed  to  the  use  of 
strong  analyses  of  the  learners  and  content.  We  feel  that  these  front-end  analyses  should  be  used  when  planning  and 
designing  future  professional  seminars. 

We  had  difficulty  measuring  learning  outcomes  during  the  seminar.  This  exemplifies  one  potential 
difficulty  in  practicing  systematic  instructional  design  for  some  professional  seminars.  When  presenters  are  invited 
guests  it  may  be  difficult  to  coordinate  the  needs  of  the  client  and  instructional  designers  with  the  presenters  long 
before  the  seminar.  The  presenter  may  not  be  able  to  prepare  ahead  of  time,  and  may  rely  more  on  his  or  her 
expertise  than  on  instractionally  sound  materials.  This  was  certainly  the  case  for  the  National  Security  Policy 
seminar. 

Another  issue  we  encountered  during  Phase  2 was  the  lack  of  perceived  benefit  of  the  discussion-based 
strategy.  Our  needs  assessment  suggested  that  this  format  could  be  very  beneficial.  However,  our  observations  and 
the  post-seminar  questionnaire  indicated  that  this  potential  may  not  have  been  achieved.  Many  studies  support  the 
premise  that  interactive  instructional  delivery  methods  can  benefit  learning,  therefore  we  believe  that  our  lack  of 
results  may  be  a result  of  poor  support  for  the  discussions.  We  could  have  provided  the  small  groups  with  additional 
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structure  about  how  to  answer  the  discussion  questions,  how  to  work  together  collaboratively,  or  how  to  solve  ill- 
structured  problems.  Some  form  of  problem-solving  scaffold  may  have  been  beneficial.  Once  again,  providing  this 
kind  of  instructional  support  and  guidance  could  be  potentially  difficult  to  implement  in  a professional  seminar  if 
instructional  designers  do  not  carefully  coordinate  with  presenters. 

Overall  Conclusions 

In  this  study,  we  sought  to  answer  questions  about  seminar  attendees’  attitudes  and  expectations,  optimal 
instructional  delivery  methods,  and  evaluation  methods.  We  briefly  provide  a summary  of  how  we  answered  these 
questions,  together  with  ideas  for  future  studies. 

Seminar  attendees  appear  to  have  high  expectations  that  the  seminars  they  attend  will  be  relevant  and 
beneficial.  This  fact  suggests  that  analyses  of  seminar  attendees,  current  issues,  and  relevant  topics  are  important 
when  planning  a seminar.  It  is  possible  that  the  relevance  of  topics  is  a facilitator  of  learning  at  seminars.  We  were 
unable  to  effectively  measure  learning,  but  future  research  projects  could  investigate  whether  there  is  a relationship 
between  perceived  relevance  and  learning  by  using  more  sensitive  measurement  of  the  factors  that  comprise 
relevance. 

While  typical  seminar  instruction  is  predominantly  teacher-centered,  our  findings  suggest  that  attendees 
expect  and  could  benefit  from  more  student-centered,  participatory,  and  interactive  delivery  methods.  Seminar 
attendees  indicated  that  they  feel  they  benefit  from  active  participation  and  discussion  in  our  needs  assessment 
survey.  However,  the  development  and  implementation  of  interactive  and  participatory  instruction  may  require 
considerable  effort  in  order  to  coordinate  instructional  designers  with  seminar  presenters. 

Learning  outcomes  related  to  problem  solving  may  be  appropriate  for  seminars.  We  successfully  produced 
learning  objects  in  the  form  of  individual  responses  to  discussion  questions,  while  also  using  an  interactive  delivery 
strategy.  Future  studies  should  further  refine  the  development  and  implementation  of  different  interactive  delivery 
and  evaluation  methods. 
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Abstract 

Knowledge  management  serves  to  capture,  store,  and  organize  knowledge  and  experiences  within  corporations  so  it 
is  available  for  future  reference.  Perhaps  the  most  difficult  part  of  knowledge  management  is  identifying  the  tacit 
knowledge  within  an  organization.  Storytelling  possesses  great  potential  as  a tool  to  uncover  tacit  knowledge  within 
an  organization,  yet  little  has  been  done  to  synthesize  literature  related  to  storytelling  in  knowledge  management. 
Research  questions  to  be  answered  include:  What  is  knowledge  management?  Why  would  knowledge  management 
benefit  from  the  use  of  storytelling?  And  how  can  storytelling  be  implemented  into  knowledge  management  to 
uncover  tacit  knowledge?  Examples  of  success  stories  of  storytelling  in  knowledge  management  will  be  presented 
and  recommendations  for  future  research  will  be  provided. 

Introduction 

Could  the  application  of  storytelling  be  a useful  component  of  knowledge  management?  If  so,  why  would 
storytelling  be  a good  tool  in  knowledge  management?  How  can  storytelling  be  used  in  knowledge  management? 
Knowledge  management  is  essentially  a teaching-learning  interaction.  Storytelling  can  be  broadly  defined  as  orally 
communicating  ideas,  beliefs,  personal  histories,  and  life-lessons  (Groce,  2004).  It  is  one  of  the  most  prevalent 
forms  of  communication  and  it  possesses  great  potential  as  a teaching-learning  tool.  It  is  no  wonder  that  it  is 
currently  being  used  in  some  companies  as  a knowledge  management  tool. 

According  to  Arlene  Dohm,  a writer  for  the  U.S.  Department  of  Labor’s  Monthly  Labor  Review,  there  are 
some  64  million  baby  boomers  (over  40  percent  of  the  U.S.  work  force)  who  will  likely  retire  in  large  numbers 
within  the  next  decade.  One-half  of  companies  interviewed  by  The  Conference  Board,  a business  membership  and 
research  organization,  “feel  that  the  departure  of  mature  workers  presents  potential  knowledge  vulnerabilities.  About 
one-third  have  conducted  workforce-planning  studies  and  identified  potential  knowledge  areas  where  they  could  be 
vulnerable.  One-half  of  those  interviewed  have  some  form  of  mentoring  program  in  place  to  share  and  transfer 
knowledge”  (“America’s,”  2005). 

The  following  is  a review  of  literature  relating  to  the  what,  why,  and  how  of  storytelling  in  the  management 
and  transference  of  knowledge  within  organizations.  Examples  are  provided  of  the  success  of  the  use  of  story  at 
NASA,  EduTech,  Bristol-Myers  Squibb,  GE,  Disney,  Nike,  Wendy’s,  FedEx  and  English  Nature  to  promote 
knowledge  transfer.  The  literature  reviewed  was  analyzed  and  compared  according  to  relevance,  quality,  reliability 
of  sources,  validity  of  information,  trends  and  issues,  and  credibility  of  author’s  research/information. 

There  are  some  trends  regarding  the  use  of  story  in  knowledge  management  to  foster  the  sharing  and 
transfer  of  tacit  knowledge.  Besides  fostering  inter-generational  communication,  knowledge  management  can  be 
used  to  improve  the  meaning  of  information  in  business.  According  to  recent  theories,  backed  up  by  relatively  new 
medical  technologies,  storytelling  is  a natural  way  of  learning,  and  more  specifically  a more  natural  way  for  the 
brain  to  process  information.  Of  course,  there  are  some  skeptics  who  question  the  real  value  of  stories  in  the 
workplace,  but  the  literature  shows  that  companies  who  are  using  stories  are  finding  it  very  valuable  for  the 
management  and  transfer  of  knowledge. 


What:  Storytelling  and  Knowledge  Management 

There  has  been  an  ongoing  interest  in  storytelling  as  a component  in  knowledge  management  over  the  last 
few  years,  but  it  has  never  really  become  a major  focus.  Storytelling  has  been  touted  as  the  best  way  to  make  the 
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leap  from  information  to  knowledge,  and  as  the  best  way  to  capture  and  transfer  tacit  knowledge  (Reamy,  2002). 
“Storytelling  is  the  skilled  delivery  of  stories  used  to  present  anecdotal  evidence,  clarify  a point,  support  a point  of 
view  and  crystallize  ideas.  A story  can  present  material  that  research  data  can  not.  Stories  use  verbal  pictures  to 
spark  interest,  add  variety,  and  change  the  pace  of  a discussion.  Stories  make  dull  speeches  sparkle.  Storytelling  is 
the  connecting  device  between  data  and  reality.  Stories  can  share  a "truth"  that  data  cannot.  Knowledge  managers 
use  storytelling  as  a device  and  tool  for  sharing  knowledge”  (Stuhlman,  n.d.). 

To  say  that  there  are  several  definitions  of  knowledge  management  would  be  an  understatement.  Denning 
(2000)  says,  “there  is  no  agreed  upon  definition  of  knowledge  management,  even  among  practitioners.”  In  a CIO 
Magazine  article  on  the  ABC’s  of  knowledge  management  (KM)  Meridith  Levinson  writes,  “Succinctly  put,  KM  is 
the  process  through  which  organizations  generate  value  from  their  intellectual  and  knowledge -based  assets.  Most 
often,  generating  value  from  such  assets  involves  codifying  what  employees,  partners  and  customers  know,  and 
sharing  that  information  among  employees,  departments  and  even  with  other  companies  in  an  effort  to  devise  best 
practices.”  Upon  “Googling”  the  term,  three  pages  worth  of  definitions  return,  Vicki  Powers  (2004)  used  a definition 
that  in  a KM  World  article,  “[Knowledge  management  is  - because  of  people’s  experiences  - a variety  of  things 
that  fall  under  that  broad  umbrella  of  KM,  including  knowledge  capture,  knowledge  stories,  communities, 
Knowledge  Desktop  and  codifying  documents.”  This  review  will  synthesize  several  resources  to  describe  how  the 
experiences  of  people  that,  according  to  Powers  (2004),  can  be  called  “knowledge  stories”  that  can  be  used  within 
an  organization  as  knowledge  management  tools. 

“To  some  people  - people  in  business,  people  in  management,  people  running  public  sector  organizations  - 
storytelling  might  seem  like  an  odd  subject  to  be  talking  about  at  all.  The  thought  that  narrative  and  storytelling 
might  be  important  ideas  in  organizational  thinking  in  the  coming  century  might  seem  even  odder”  (Brown  et  ah, 
2004).  It  may  come  as  no  surprise,  therefore,  that  there  is  some  controversy  surrounding  storytelling  as  a method  of 
transferring  knowledge  in  business.  The  idea  of  storytelling  being  used  in  business  “causes  some  disquiet  among 
practical  businesspeople,  information  specialists,  and  even  many  in  knowledge  management”  (Reamy,  2002).  The 
images  of  groups  of  businesspeople  squatting  around  a campfire  swapping  stories  and  wasting  time  as  executives 
tell  bad  stories  and  unload  their  fears  and  anxieties  does  not  appeal  to  some  knowledge  managers  (Reamy,  2002). 
This  criticism  is  likely  a result  of  a misunderstanding  of  what  storytelling  can  do  for  knowledge  management  in 
companies,  and  more  importantly,  the  techniques  involved  in  implementing  storytelling  as  a tool.  For  example,  Seth 
Kahan  uses  a technique  called  “jumpstart  storytelling,”  which  “harnesses  diverse  ideas  and  fosters  discussion  about 
what’s  important  - constructive  conversations”  (Dysart,  2004). 

“. . . [T]he  potential  of  storytelling  as  a viable  teaching-learning  tool  can  be  recognized  only  if  its  inherent 
nature  is  clearly  understood”  (Roney,  1996).  There  is  some  research  that  supports  this  “inherent  nature”  of 
storytelling.  This  research  suggests  that  people  learn  best  with  stories  because  “organization  of  information  in  story 
form  is  a natural  brain  process”  (Caine  and  Caine,  1991).  As  educators  then,  knowledge  managers  who  use 
storytelling  are,  as  Bruce  Perry  (2000)  puts  it,  practically  neuroscientists  because  the  brain  is  more  receptive  to 
information  in  story  form.  Storytelling  is  a useful  tool  for  knowledge  management  because  people  learn  things  easier 
via  stories  - they  convey  meaning,  convey  tacit  knowledge,  and  are  a natural  way  of  learning  and  interpreting 
information. 


Why:  Benefits  of  Storytelling  in  Knowledge  Management 

Stories  put  meaning  into  information,  stories  convey  tacit  knowledge  more  easily,  and  stories  are  a more 
natural  way  of  learning  than  the  usual  charts  and  facts  that  are  presented  in  knowledge  management.  There  are  a 
number  of  benefits  in  using  storytelling  in  knowledge  management.  Several  leading  companies  are  “turning  to 
mature  workers  to  ensure  future  growth  and  productivity.  These  companies  recognize  that  a maturing  workforce  can 
positively  impact  customer  satisfaction  and  profitability,  but  not  without  effective  initiatives  designed  to  make  it 
easier  for  different  generations  of  workers  to  work  better  together,”  (America’s,”  2005).  This  inter-generational 
cooperation  can  be  accommodated  with  the  use  of  story  to  foster  communication  and  knowledge  transfer. 

To  answer  the  question  of  why  knowledge  management  should  incorporate  storytelling  Reamy  (2002)  says, 
“Humans  have  been  telling  stories  as  not  only  a form  of  entertainment,  but  as  a way  to  make  sense  of  the  world  for  a 
very  long  time-probably  almost  as  long  as  they  have  had  language.  So  it  is  not  a surprise  that  we  continue  to  use  this 
powerful  medium  in  the  corporate  environment.”  Stories  are  used  in  three  ways,  according  to  Reamy,  (2002):  (1)  in 
informal  education  and  training  where  stories  provide  guidance  and  lessons  in  the  advanced  or  more  sophisticated 
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application  of  basics,  (2)  the  foundation  for  many  formal  and  informal  communities  that  form  within  an  enterprise, 
and  (3)  what  Stephen  Denning  calls  “springboard  stories,”  which  are  used  to  create  a new  paradigm  by  not  only 
introducing  new  ideas,  but  getting  people  to  actively  promote  the  new  idea. 

Meaningful  Information:  Facts  vs.  Stories 

Stories  have  been  told  since  the  beginning  of  time-from  the  walls  of  cave  men  and  the  tombs  of  Egypt  to 
modern-day  dinner  table  and  campfire  stories.  “People  primarily  think  narratively  rather  than  analytically  or 
argumentatively”  (Stone,  n.d.).  When  people  tell  a story  it  is  because  it  means  something  to  them  (the  teller)  and 
they  think  that  it  might  mean  something  to  the  audience  (the  listeners).  Stories  as  metaphors  are  intrinsic  to  the 
construction  of  new  knowledge  and  at  the  heart  of  the  acquisition  of  felt  meaning  (Caine  and  Caine,  1991). 

Although  some  skepticism  surrounds  the  idea  of  stories  in  business,  it  should  not  be  discounted  altogether;  and 
neither  should  executives  discourage  people  from  telling  stories  in  the  workplace.  Stone  (n.d.)  says  that  in  order  to 
understand  storytelling  as  a tool,  one  must  recognize  that  people  are  “already  telling  stories  all  the  time”  at  work. 

Stories  make  information-especially  large  amounts  of  information-easy  to  understand.  Storytelling  as  a 
means  of  knowledge  management  creates  experience  in  context  and  nurtures  creativity  (Schwartz,  2004).  In  2004 
West  et  al.  cite  Pennington  and  Hastie  (1991)  and  Schank  (1990)  as  saying  that  past  research  demonstrates  that  one 
of  the  aspects  that  make  stories  effective  is  that  they  are  easily  stored  in  memory.  “. . . [N  jarrativc  communicates  a 
more  meaningful  sense  of  reality  than  the  abstract  and  summative  formations  of  most  business  plans”  (Stone,  n.d.). 
When  it  comes  to  purely  listed,  factual  information.  Swap  et  al.  (2001)  say  that  stories  promote  elaborations  and 
connections  to  the  listener’s  personal  experience,  evoke  clear  visual  images,  are  more  memorable  and,  hence,  “more 
effective  carriers  of  knowledge  than  less  vivid,  purely  listed  information.”  Reamy  (2002)  also  says  that  if 
information  is  put  into  stories  as  clusters  or  chunks,  it  is  easier  to  pay  attention  to  the  information  being  given. 

On  his  website,  Steve  Denning  says  the  origin  of  his  interest  in  organizational  storytelling  “was  simple: 
nothing  else  worked.”  As  a manager  in  the  World  Bank  in  1996,  he  “had  been  trying  to  communicate  the  idea  of 
knowledge  management  and  to  get  people  to  understand  and  to  implement  it.  At  that  time  in  that  organization, 
knowledge  management  was  a strange  and  generally  incomprehensible  idea.  1 used  the  traditional  methods  of 
communicating  with  no  success.  I gave  people  reasons  why  the  idea  was  important  but  they  didn't  listen.  I showed 
them  charts  and  they  just  looked  dazed.  In  my  desperation,  I was  willing  to  try  anything  and  eventually  1 stumbled 
on  the  power  of  a story”  (“Steve  Denning,”  n.d.)  Denning  continues  on  the  success  of  story  at  World  Bank,  "In 
1996  in  the  World  Bank,  story  had  helped  galvanize  staff  and  managers  to  imagine  a different  kind  of  future  for  the 
organization  and  to  set  about  implementing  it.  Denning  continued  to  use  stories  to  reinforce  and  continue  the 
change.  The  efforts  were  successful:  by  2000,  the  World  Bank  was  benchmarked  as  a world  leader  in  knowledge 
management." 

According  to  Kathleen  Melymuka  (2004),  “Stephen  Denning  discovered  a powerful  leadership  tool: 
storytelling”  that  he  found  “often  succeeded  in  inspiring  and  motivating  people  when  cold,  hard  logic  failed.” 
Escalas  and  Bettman  and  McGregor  and  Holmes  note  that  “memories  are  more  easily  stored  and  retrieved  in  story 
form,  particularly  when  they  encompass  a goal,  action,  and  some  kind  of  resolution  (qtd.  in  West  et  al.,  2004). 
Storytelling  has  great  potential  for  making  knowledge  meaningful  in  business.  Swap  et  al.  explain  three  ways  in 
which  stories  can  be  used: 

■ the  availability  heuristic, 

According  to  Tversky  and  Kahneman  (qtd.  in  Swap,  et  al.,  2001),  “. . .when  an  event  is  made  more 
available  from  memory,  there  is  a strong  tendency  to  believe  it  is  more  likely  to  occur  or  to  be  true 
elaboration, . . . 

■ elaboration, 

[to]. . .the  extent  that  people  reflect  upon  and  integrate  information  with  what  they  already  know, 
they  will  remember  it  better.” 

■ episodic  memory, 

“Stories  are  clearly  episodic  in  nature.”  Episodic  memories  come  from  memorable  experiences. 
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In  summary,  availability  heuristic,  elaboration,  and  episodic  memory  are  good  ways  to  use  stories  in 
knowledge  management  to  turn  factual  information  into  something  that  is  memorable;  stories  can  be  attached  to 
something  people  already  know;  and  they  can  create  an  experience  that  is  easy  to  remember. 

Storytelling  and  Tacit  Knowledge 

Tacit  knowledge  is  “knowledge  that  we  do  not  know  that  we  know”  (Hughes,  2002).  According  to  Nonaka 
and  Takeuchi,  “. . .knowledge  of  greatest  value  to  an  organization-expert  knowledge-is  often  tacit”  (qtd.  in  Hughes, 
2002).  Tacit  knowledge  can  be  difficult  both  to  convey,  and  to  acquire  in  most  organizations.  It  is  important  to 
acquire  tacit  knowledge  from  those  people  who  have  seniority  and  who  have  been  with  the  company  or  in  the 
business  for  some  time.  Storytelling  is  an  effective  way  of  getting  tacit  knowledge  out  so  others  can  use  it  and  refer 
to  it.  Acquisition  of  knowledge  involves  experience — which  storytelling  has  always  existed  to  provide  (Schwartz, 
2004).  Stories  of  experiences  or  trials  and  errors,  for  example,  would  likely  benefit  those  who  have  not  had  much 
opportunity  to  acquire  such  experiences  or  the  knowledge  that  results  from  those  experiences.  “Storytelling  is  a way 
of  capturing  what’s  unique,  and  what’s  unique  per  individual  is  tacit  knowledge”  (Post,  2002).  It  is  possible  that 
using  storytelling  to  convey  tacit  knowledge  can  improve  an  organization  because  “one  aspect  of  knowledge 
management  is  about  sharing  experiences  so  others  can  learn  from  them,  and  then  using  that  learning  to  improve  the 
organization”  (Sumner,  2005).  According  to  Harold  Rosen,  “...once  set  loose,  the  storytelling  impulse  enables  the 
most  unlikely  people  to  deploy  unsuspected  linguistic  resources  and  strategies”  (qtd.  in  Rooks,  1998).  Expert,  tacit 
knowledge  within  an  organization  can  easily  be  conveyed  through  storytelling. 

Storytelling:  Natural  Way  to  Learn 

The  intent  of  knowledge  management  is  to  educate  people  for  the  greater  good  of  the  organization  and, 
when  educating  people,  it  is  important  to  understand  how  they  learn.  Research  says  the  brain  works  by  detecting 
patterns  in  information,  which  is  why  stories  are  so  useful  in  the  teaching-learning  interaction.  “One  of  the  brain’s 
best  tricks  is  to  extract  meaningful  patterns  from  confusion”  and  storytelling  is  an  “excellent  way  of  accomplishing 
this  task”  (Liston,  1994).  Learning  is  a brain-based  activity  and  factual  information  causes  the  brain  to  fatigue  more 
quickly  because  it  is  not  the  brain’s  most  “natural”  way  of  learning  (Perry,  2000).  Stories  give  meaning  to 
information  (as  mentioned  before)  because,  as  Roger  C.  Schank  says,  the  “human  memory  is  story-based”  (qtd.  in 
West,  Huber,  and  Sam  Min,  2004).  If  the  question  is  why  stories  work  so  well  in  knowledge  management,  then  the 
answer-simply-is  that  “our  brains  seem  to  be  wired  to  easily  and  almost  automatically  organize  information  into 
stories”  (Reamy,  2002).  Liston  (1994)  relates  brain  function  to  learning.  She  says  when  we  attempt  to  educate 
people  “we  extract  the  ‘facts’  from  the  plethora  of  information,  in  an  effort  to  clarify  the  ‘important’  information; 
conversely,  it  seems,  that  an  understanding  of  brain  functioning  indicates  we  would  better  enable  students  to  learn  if 
we  presented  masses  of  information  and  allowed  the  learner  to  detect  patterns.” 


How:  Tips  for  Success 

Stories  serve  as  metaphors  because  they  make  information  more  meaningful  and  understandable  and 
transfer  the  difficult-to-uncover  tacit  knowledge  within  organizations  by  appealing  to  the  natural  learning  process  of 
the  brain.  “Metaphors  allow  an  entry  point  into  an  issue  that  might  seem  too  intimidating  to  confront  head  on”  (Post, 
2002).  The  final  question  is  how  to  implement  storytelling  as  a knowledge  management  tool. 

On  his  website,  Stephen  Denning  (2000)  summarizes  his  technique  called  “the  Springboard  Story,”  which 
is  a story  that  “enables  a leap  in  understanding  by  the  audience  so  as  to  grasp  how  an  organization  or  community,  or 
complex  system  may  change.”  This  technique  has  been  called  a “powerful  communication  tool”  by  the  readers  of 
his  book,  The  Springboard.  Reamy  (2002)  recommends  organizing  a central  group  to  administrate  the  storytelling 
and  to  create  a reward  system  for  employees  who  submit  stories.  Different  kinds  of  stories  can  be  used  for  different 
situations  and  Melymuka  (2004)  provides  a chart  for  creating  stories  to  match  situations  in  the  information 
technology  world,  which  could  very  easily  be  applied  to  knowledge  management.  The  chart  explains  that  for  certain 
purposes,  stories  can  be  used  to  invoke  specific  kinds  of  responses.  Larry  Todd  Wilson  was  cited  as  having 
identified  four  situations  where  storytelling  could  be  helpful:  new  unexpected  situations,  situations  that  require 
feelings  as  well  as  thoughts,  complex  situations,  and  situations  in  which  you  need  to  help  people  understand  ‘why’ 
(“Storytelling,”  n.d.).  Support  for  the  effectiveness  of  applying  storytelling  to  these  situations  is  provided  by 
McDaniel  (2004),  “. . .stories  are  vessels  for  storing  and  communicating  complex  ideas”  and  “. . .can  be  very  efficient 
in  helping  one  to  learn  unfamiliar  material.” 
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There  is  some  word  of  caution  when  using  stories  in  knowledge  management,  “artificially  constructed 
stories  ultimately  will  be  less  effective  than  true  ones”  (Swap,  et  al.,  2001).  In  other  words,  stories  should  not  be 
made  up  to  teach  a lesson  or  strategy-they  should  be  real  stories  from  real  experiences.  Storytelling  can  be  counter- 
productive when  the  story  told  is  not  true  (Denning,  2000).  Denning  also  cautions  that  use  of  the  Springboard  is  not 
for  all  audiences.  In  his  book,  Denning  explains  that  “not  all  stories  have  the  ‘springboard  effect’”  and  it  only 
“worked  well  with  particular  audiences.” 

Perry  offers  a technique  for  incorporating  story  into  curriculum  (or  knowledge  management)  called  the 
bob-and-weave  technique.  “Engage  [the  audience]  with  a story  to  provide  the  context.  Make  sure  this  vignette  can 
touch  the  emotional  parts  of  their  brains.  This  will  activate  and  prepare  the  cognitive  parts  of  the  brain  for  storing 
information.  Information  is  easiest  to  digest  when  there  is  emotional  ‘seasoning’  — humor,  empathy,  sadness,  and 
fear  all  make  "dry"  facts  easier  to  swallow.  Give  a fact  or  two;  link  these  facts  into  related  concepts.  Move  back  to 
the  narrative  to  help  them  make  the  connection  between  this  concept  and  the  story.  Go  back  to  another  fact. 
Reinforce  the  concepts.  Reconnect  with  the  original  story.  In  and  out,  bob  and  weave,  among  facts,  concept,  and 
narrative.”  This  technique  of  moving  back  and  forth  through  these  interrelated  neural  systems,  weaving  the 
information  together  is,  according  to  Perry,  the  most  effective  way  to  present  information. 

The  website  for  i.d.e.a.s.  Learning,  an  Integrity  Arts  and  Technology  company  who’s  core  competency  is 
storytelling,  says,  “Educators  and  trainers  have  always  known  that  you  have  to  start  with  what  a learner  knows,  and 
use  that  to  build  a bridge  to  the  new  knowledge  or  skill  you  want  them  to  acquire.  Turns  out,  that  whether  you  tell 
them  the  story  or  they  think  it  up  on  their  own,  there  is  a story  at  the  core  of  that  new  learning!  The  point  is  to  give 
the  learner  the  story  that  will  carry  them  toward  the  objectives  rather  than  leaving  it  up  to  chance.  The  story  has  to 
provide  both  the  context  for  the  new  learning,  as  well  as  the  vehicle  to  get  them  there”  (“i.d.e.a.s.  Learning,”  n.d.). 


Success  Stories 

Internal  magazines  have  been  effective  in  eliciting  tacit  knowledge  for  NASA  and  EduTech.  The 
employees  at  EduTech  have  “written  stories  about  their  experience  and  hope  that  future  teams  will  learn  from  their 
mistakes  about  ‘what  not  to  do’  by  reading  these  stories”  (Post,  2002).  “Certain  industries  are  more  concerned  with 
the  impending  “brain  drain”  stemming  from  the  withdrawal  of  some  mature  workers  from  the  workforce.  The 
technology  and  pharmaceuticals  industries  generally  express  worries  about  the  development  of  new  products  and 
services  and  anticipate  a drain  in  experienced  engineers,  key  account  sales  representatives,  and  senior  managers” 
(“America’s,”  2005). 

Bristol-Myers  Squibb  adapted  the  successful  practice  of  storytelling  “from  NASA’s  Jet  Propulsion 
Laboratory  to  provide  time  for  senior  or  tenured  people  in  the  organization  to  share  tacit  knowledge  with  others 
through  storytelling.”  (Powers,  2004).  Melinda  Bickerstaff  of  Bristol-Myers  Squibb  says,  “We  are  really  focusing  on 
tacit  knowledge-the  stuff  underneath  that  is  really  difficult  to  get  at.  A company  that  can  figure  out  how  to  do  that 
and  begin  to  share  with  others  in  the  organization  will  be  the  company  that’s  really  ahead.”  (qtd.  in  Powers,  2004). 

A growing  number  of  companies  are  using  storytelling  to  increase  employee’s  morale,  get  mission 
statements  across  the  workforce,  recruit  new  staff  members,  and  praise  existing  ones.  One  Nightime  Pediatrics 
Clinic  hired  Rick  Stone,  president  of  StoryWork  Institute  (Orlando,  FL),  to  identify  stories  that  reflect  the  core  value 
of  the  company.  StoryWork  is  a national  company  that  has  developed  training  programs  dealing  with  storytelling  for 
teambuilding  and  leadership.  The  partnership  resulted  in  the  development  of  “Nightime  Stories,”  a book  of  yarns 
told  by  staff  and  patients  of  the  hospital  (Holden,  1999). 

This  highlights  that  storytelling  has  a place  in  the  corporate  world,  and  the  CEO  of  GE,  Jack  Welsh,  could 
best  exemplify  this.  He  is  a great  storyteller  and  he  has  encouraged  his  managers  to  hone  their  storytelling  skills. 
Storytelling  is  an  effective  tool  that  could  be  used  to  direct  and  inspire  subordinates.  This  is  because  it  evokes  both 
visual  imagery  and  emotions  among  the  listener.  It  is  a fun  way  of  communicating  concepts  and  ideas  within  an 
organization  (Dennehy,  1999). 

Another  company  that  utilizes  storytelling  is  The  Walt  Disney  Company.  Bran  Ferran,  executive  vice 
president  for  Walt  Disney  Imagineering,  believes  that  the  web  is  a great  medium  for  storytelling  and  he  also  believes 
that  each  job  is  primarily  a storytelling  job.  He  asserts  that  chief  information  officers  (CIOs)  could  accomplish  their 
jobs  more  easily  and  efficiently  if  only  they  could  become  more  effective  storytellers.  He  believes  that  IT 
departments  could  serve  as  a storage  space  for  memories;  information  technology  should  be  used  to  convey  the 
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company’s  sense  of  community,  history  and  accomplishments.  Companies  should  also  realize  that  the  Internet  could 
bring  numerous  benefits  and  building  an  intranet  would  allow  a company  to  extend  these  benefits  within  the 
organization  (Jahnke,  1998). 

At  3M,  strategic  narratives  and  storytelling  are  a better  alternative  to  using  bullet  points  in  strategic 
planning.  Planning  by  narratives  is  a lot  like  storytelling.  The  plan  must  tell  how  the  company  can  overcome 
obstacles  and  win.  The  conclusion  requires  a logical,  concise  argument  that  is  specific  to  the  situation  and  leads  to 
desired  outcomes.  The  story  should  include  the  industry’s  economic  factors,  its  key  success  factors,  and  the  force 
that  drives  the  change.  The  story  should  introduce  a dramatic  conflict  and  should  reach  a solution  that  ends  the  story 
in  a satisfying  manor  (Shaw,  Brown,  and  Bromiley,  1998). 

At  an  express  carrier  company  in  Great  Britain,  Parcelforce  Worldwide,  senior  executives  have  developed  a 
corporate  story  that  is  designed  to  communicate  the  business  plan  and  motivate  its  personnel  to  meet  the  firm’s 
objectives.  The  storytelling  approach  would  humanize  the  strategy,  make  the  business  plan  easy  to  remember,  and 
help  generate  practical  ideas  that  would  help  improve  productivity  (Jackson  and  Esse,  2006).  Organizations  use 
words  to  define  themselves  to  the  market  and  also  to  their  own  organization.  In  2006,  Blair  noted  a few  companies 
that  do  so.  FedEx  has  a long  mission  statement  that  is  conveyed  through  words  and  storytelling.  Wendy’s  and  Nike 
use  words  and  storytelling  as  a way  to  convey  their  mission  statements.  If  you  are  convinced  that  you  would  like  to 
use  stories  in  your  company,  Blair  notes  a few  elements  to  keep  in  mind: 

■ Be  clear  about  what  you  want  to  explore  through  story 

■ Make  sure  that  everyone’s  story  can  be  told 

■ Decide  how  you  will  elicit  the  stories 

■ Consider  how  some  of  the  stories  can  be  told  to  the  whole  group 

■ Take  time  to  process  the  stories 

■ Consider  the  types  of  stories  you  want 

■ Believe  in  the  power  of  the  stories 

Storytelling  improved  the  ability  of  English  Nature  to  share  and  reuse  knowledge  from  its  constituents 
through  the  use  of  reverse-storytelling  and  listening  skills  when  working  with  farmers,  landowners  and  partner 
organizations.  English  Nature  armed  with  an  ever-growing  arsenal  of  teaching  and  springboard  stories  understands 
the  power  of  the  story  (Donaldson,  2005).  Harris  and  Barnes  (2005)  say  stories  can  entertain,  teach,  delight,  frighten, 
or  inspire.  They  are  tools  of  leadership.  Stories  can  reach  across  boundaries  of  culture,  profession,  and  age. 
Storytelling  is  effective  in  leadership.  If  a company  is  trying  to  improve  conditions,  to  inspire  employees  to  work 
harder  and  longer,  or  to  communicate  a complex  idea  in  a clear  and  powerful  way,  the  solution  is  to  tell  stories. 

Strategic  management  also  uses  storytelling  as  a tool  to  enhance  communication.  Applied  correctly, 
storytelling  can  do  more  than  inform  employees  about  an  initiative;  it  can  add  context  and  encourage  acceptance. 
According  to  Snowden  (2001)  story  is  a very  powerful  tool;  it’s  the  most  sustainable  of  communication  techniques. 
It’s  not  just  about  communication,  it’s  about  understanding  a corporate  culture  and  providing  a means  whereby  real 
learning  and  innovation  takes  place. 


The  Verdict 

Could  the  application  of  storytelling  be  a useful  component  of  knowledge  management?  Yes,  storytelling 
could  be  very  useful  in  knowledge  management,  and  for  some  companies  has  proven  to  be  a very  worthwhile 
investment.  Why  would  storytelling  be  a good  tool  in  knowledge  management?  Storytelling  could  be  useful  in 
knowledge  management  because  it  makes  information  meaningful,  makes  tacit  knowledge  more  explicit,  and  puts 
information  into  more  natural  learnable  chunks.  How  can  storytelling  be  used  in  knowledge  management? 
Committees  can  be  assembled  in  organizations  with  the  objective  of  getting  the  tacit  knowledge  and  making  it  more 
explicit.  Storytelling  techniques  can  be  used  to  convey  the  usual  facts  and  figures  to  make  them  more  meaningful 
and  easier  to  learn. 

Current  research  says  that  storytelling  could  potentially  have  a very  valuable  impact  on  business,  but  more 
companies  need  to  try  using  it  and  report  on  their  successes  and/or  difficulties  with  it.  Although  there  were  no 
reports  of  unsuccessful  attempts  at  implementing  stories  into  knowledge  management,  it  cannot  be  assumed  that 
there  were  no  unsuccessful  attempts.  Storytelling  as  a means  to  understand  the  world  and  to  spread  information  has  a 
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long  and  rich  history,  it  is  and  it  is  highly  likely  we  will  see  this  trend  in  learning  and  teaching  making  headlines  as 
the  latest  and  greatest  tool  in  knowledge  management. 
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Cross  Cultural  Considerations  for  Online  Teaching  and  Learning 

Seung-hee  Lee 
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Indiana  University 

Introduction 

From  a social  constructivist  perspective,  learning  and  culture  are  interdependent.  Cultural  contexts 
influence  how  students  perceive,  understand,  and  interpret  knowledge.  Thus,  learning  in  different  countries 
requires  different  intellectual  capabilities  and  different  cultural  approaches  (Hofstede,  1986). 

In  general,  an  online  environment  is  designed  to  open  to  students  who  are  physically  located  in 
different  sites  or  countries.  An  online  environment  requires  group  learning  with  which  students  communicate 
each  other  to  build  collective  knowledge.  The  impact  of  multi-national  cultures  can  be  a big  variable  to 
determine  the  success  of  online  teaching  and  learning.  Language  barriers,  different  communication  styles,  or 
interactions  with  peer  students  from  different  countries  would  be  a significant  challenge  for  students 
(Gunawardena,  Nolla,  Wilson,  Lopez-Islas,  Ramirez- Angel,  & Megchun-Alpizar,  2001).  Further,  cultural 
backgrounds  of  students  may  enable  or  hinder  building  their  learning  experiences  in  an  online  environment. 

With  consideration  of  culture  factors  as  student  characteristics,  the  more  knowledge  on  cultural 
diversity  associated  with  learning  instructors  have,  the  more  appropriate  support  would  be  available  for 
building  meaningful  learning  experiences  for  culturally  diverse  students.  Thus,  instructors  and  online 
educational  practitioners  need  to  be  prepared  to  understand  cultural  differences  of  students  and  address  how 
cultural  differences  might  impact  on  online  teaching  and  learning  (Cifuentes  & Shih,  2001).  In  this  sense,  the 
focus  of  this  study  is  to  review  cultural  differences  from  pedagogical  and  technological  perspectives. 

Frameworks  for  understanding  cultural  differences 

Culture  is  defined  as  “a  set  of  learned  values,  attitudes,  and  approaches  shared  by  a group  of  people” 
(Duarte  & Snyder,  1999,  pp.  54).  Over  the  learning  process,  students  expose  their  cultural  orientations  which 
have  been  embeded  in  their  ways  of  thinking,  behaviors  with  or  without  intention.  They  develop  the  processes 
of  influencing  the  cultural  perspectives  and  being  influenced  by  their  cultural  values  interchangeably.  Thus,  it 
is  said  that  teaching  and  learning  is  essentially  the  transmission  of  cultures  and  culture  acquisition  from  a 
perspective  of  instructors  and  students  (Wolcott,  1999).  Accordingly,  it  can  be  said  that  “culture  and  learning 
are  interwoven  and  inseparable”  (McLoughlin,  1999,  p.  232). 

Why  does  the  cultural  issue  matter  in  an  online  environment?  An  online  environment  presents  the 
nature  of  complex  dynamics.  There  are  ambiguity  of  presences  in  virtual  spaces  and  restricted  contextual 
clues  in  online  dialogues  and  communication  behaviors  which  little  happen  in  face-to-face  classroom  setting. 
With  cultural  diversity  added,  online  teaching  and  learning  would  be  more  challenging  than  those  of  face-to- 
face  classrooms.  McLoughlin  and  Oliver  (1999)  point  out  that  instructors  and  online  educational  practitioners 
should  consider  norm,  values,  learning  behavior  styles,  and  cognitive  preferences  of  students  in  designing 
tasks  and  activities  in  order  to  make  an  online  environment  real-world  contextual.  Simultaneously,  they 
should  be  aware  that  certain  educational  values  in  one  culture  are  culturally  inappropriate  to  anther. 
Admittedly,  culturally  appropriate  approaches  are  required  for  better  teaching  (McLoughlin  & Oliver,  1999). 

Two  theoretical  frameworks  from  Flofstede  (1986)  and  Flail  (1976)  explain  about  cultural  orientation 
of  people  from  different  nations.  Flofstede’s  framework  suggests  several  dimensions  that  determine  cultural 
differences:  power  distance,  collectivism  vs.  individualism,  femininity  vs.  masculinity,  uncertainty  avoidance, 
long  term  vs.  short  term.  On  the  other  hand,  Flail  determines  cultures  into  two  different  categories  by  high 
contextual  culture  vs.  low  contextual  culture. 

The  cross-cultural  frameworks  of  Flofstede  and  Flail  benefit  to  instructors  and  online  educational 
practitioners  to  draw  a comprehensive  picture  of  understanding  cultural  differences  of  international  students. 
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Based  on  this,  it  might  need  to  clarify  what  implications  and  challenges  cultural  factors  give  to  those  who  are 
willing  to  understand  how  multi-cultural  students  act,  react,  or  behave  in  an  online  environment. 


Considerations  for  online  cross-cultures 

Bigg  (1999,  cited  in  Bamford,  Marr,  Pheiffer,  & Weber-Newth,  2002)  points  out  that  social-cultural 
adjustment,  language,  and  specific  learning  and  teaching  problems  are  the  major  problem  areas  of  students 
with  different  cultural  orientations.  An  online  learning  environment  is  not  exceptional  and  it  can  face  even 
more  complex  problems.  Here  this  study  suggests  understanding  how  cultural  differences  influence  student 
learning,  and  understanding  the  relationship  of  cultural  differences  with  technology  preferences. 

Understanding  cultural  differences  in  student  learning : It  is  mandatory  that  instructors  and  online 
educational  practitioners  should  reflect  instructional  values  and  beliefs  into  their  instructional  design  of  online 
teaching  and  learning  (Bentley,  Tinney,  & Chia,  2005).  At  the  same  time,  instructors  should  clarify  students’ 
cultural  diversity  and  further  understand  how  they  communicate  and  engage  in  online  learning.  For  instance, 
Massey,  Montoya-Weiss,  Hung,  and  Ramesh  (2001)  pointed  out  that  students  in  collectivistic  cultures  rely 
upon  high  context  communications.  They  usually  spend  enough  time  on  making  casual  conversations  and 
interactions  to  create  a feeling  of  comfort  before  addressing  task-oriented  discussions.  In  this  case,  warming- 
up  socializing  activities  (e.g.,  coffee  house  activity,  team  building  activities,  etc.)  assist  students  to  get  to 
know  each  other  as  a way  of  stimulating  task-related  dialogues.  Students  in  high  contextual  cultures  tend  to 
avoid  high  uncertainty  and  prefer  to  know  whether  others  understand  them  and  whether  they  can  understand 
others  under  the  same  communication  circumstance.  They  tend  to  reach  decisions  through  indirect 
communication  with  bearable  degree  of  vagueness  to  avoid  conflicts. 

As  Palloff  and  Praff  (2005)  indicate,  what  is  important  from  a pedagogical  perspective,  is  to 
recognize  the  differences  of  cultures,  understand  different  communication  styles  and  attitude  toward  online 
teaching  and  learning  environments.  It  is  important  to  develop  both  ‘culturally-general’  and  ‘culturally- 
specific’  instructional  strategies  when  designing  and  facilitating  online  courses. 

Understanding  cultural  preference  on  technology > use : All  the  teaching  and  learning  activities  in 
online  environments  are  conducted  via  technologies.  Cultural  orientations  may  determine  students’  reactions 
and  preferences  on  particular  technological  tools.  For  instance,  rich  technological  tools  reduce  a feeling  of 
solitude  and  uncertainty  which  frequently  happen  to  online  environments.  Students  in  collectivist  cultures  feel 
comfortable  with  face-to-face  interactions  since  synchronous  interactions  are  effective  in  enhancing  a sense  of 
belonging  (Duarte  & Snyder,  1999).  Students  with  high  contextual  cultures  tend  to  learn  better  knowledge  and 
information  in  visual  presentations.  Video  conferencing  or  face-to-face  meeting  can  serve  to  clearly  deliver 
important  messages  and  social  presence  to  students. 

On  the  other  hand,  lean  technological  tools  lack  immediate  feedback,  uses  a single  channel,  and 
support  limited  cues  (Workman,  Kahnweier,  & Bommer,  2003)  and  they  send  fewer  communicational  cues  or 
provide  slower  feedback.  Email,  one  of  lean  technological  tools  can  be  fully  utilized  by  students  under  low 
contextual  cultures  by  delivering  explicit  words,  arguments,  debates.  Thus,  instructors  should  aware  that  the 
dominant  use  of  a particular  technological  tool  may  be  comfort  for  some  students,  but  a cognitive  burden  for 
others. 


Conclusion 

A variety  of  online  courses  across  different  disciplines  have  been  delivered  to  international  students 
over  the  years.  While  current  research  studies  recognize  the  importance  of  cultural  understanding  in  online 
learning  environments,  technologies  have  long  being  used  under  instructional  intentions  with  little  regard  for 
the  cultural  orientation  of  students.  Further,  little  has  been  done  to  examine  the  relationship  of  pedagogy, 
technology,  and  cultural  factors  in  online  environments.  It  is  fortunate  to  say  that  there  is  a growing  attention 
to  cultural  issues  related  to  online  teaching  and  learning  recently. 

It  is  obvious  that  online  teaching  and  learning  is  not  an  easy  practice.  It  is  even  harder  than  face-to- 
face  classroom  setting  and  it  requires  more  time  and  effort  than  expected.  A “culture-technology  fit”  (Massey 
et  ah,  2001,  pp.  84)  with  a pedagogical  perspective  may  be  new  to  all  students,  instructors,  and  online 
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educational  practitioner.  However,  it  would  be  our  responsibility  to  dig  it  out  in  order  to  develop  cross- 
cultural  online  learning  environments.  It  is  expected  that  the  initial  step  forward  this  topic  contributes  to 
increase  the  quality  of  online  teaching  and  learning  in  the  end. 
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Introduction 

A business  school  puts  high  values  on  ‘theories  into  practices’  in  which  students  conduct  real-world  tasks 
as  much  as  or  more  than  any  other  fields.  With  extensive  application  of  knowledge  and  skills,  case -based  learning 
has  been  regarded  as  a powerful  pedagogical  tool  across  business  disciplines.  A case  plays  a vital  role  in  areas  such 
as  accounting,  business  law,  management,  marketing,  finance,  and  information  systems. 

Some  benefits  of  the  case -based  learning  method  include  the  development  of  connections  between 
theoretical  and  practical  knowledge,  student  engagement,  reasoning,  and  reflective  decision-making  (Lundeberg, 
Levin,  & Harrington,  1999;  Wang  & Bonk,  2001).  Such  skills  are  necessary  for  MBA  students  to  be  ready  to 
perform  well  in  real-worlds,  conduct  ill-structured  tasks,  and  be  able  to  make  decisions  based  on  constantly 
changing  sources  of  information. 

However,  despite  the  growing  popularity  of  using  the  case-based  approach  in  distance  education  of 
business,  Oram  (1998)  indicated  that  most  online  case  studies  adopt  traditional  text-based  materials,  resources,  and 
approaches.  There  is  little  agreement  about  such  fundamental  issues  as  how  to  support  online  case -based  activities, 
and  what  to  assess  the  learning  performance  of  online  case-based  activities.  It  is  unclear  how  online  MBA  courses 
have  been  designed  to  support  students’  learning  experiences  with  case-based  activities. 

This  study  focuses  on  the  review  on  current  states  of  online  case -based  learning  as  a way  of  MBA 
professional  development.  It  explores  the  perceptions  of  instructors  and  students  on  actual  implementations  of  case- 
based  learning  within  an  online  MBA  program,  from  both  pedagogical  and  technological  perspectives.  The  results  of 
the  study  are  expected  to  enable  instructors  to  better  understand  the  key  components  of  online  case-based  learning  to 
understand  how  to  create  meaningful  environments. 


Literature  review 

Case -based  learning  in  a business  school  context  is  defined  as  a method  that  “involves  studying  actual 
business  situations,  written  as  an  in-depth  presentation  of  a company,  its  market,  and  its  strategic  decisions,  in  order 
to  improve  a student's  problem-solving  ability.  Cases  typically  investigate  a contemporary  issue  in  a real-life 
context.  There  are  multiple  issues  to  consider  and  many  correct  or  viable  alternatives  to  solve  the  case  issues  are 
presented"  (Helms,  1999,  pp.71).  This  definition  echoes  that  case-based  learning  can  help  students  prepare  to  deal 
with  real-world  problems  that  they  will  face  at  the  time  when  they  leave  an  academic  environment,  and  ultimately  to 
find  success  within  a specific  business  organization  or  profession  by  using  these  previously  learned  skills  and 
experiences. 

With  the  case  study  method,  students  are  prompted  to  recall,  anchor,  and  integrate  their  prior  experiences  to 
analyze  cases  and  to  explore  solutions  to  identified  problems  in  the  cases  through  discussions,  reasoning,  reflection, 
and  decision  making  (Lundeberg,  Levin,  & Harrington,  1999;  Wang  & Bonk,  2001).  Cases  that  present  real  or 
hypothetical  problems  and  scenarios  can  prompt  deep  discussions  to  develop  solutions  (Benbunan-Fich  & Hiltz, 
1999).  Designing  activities  with  interesting  and  pedagogically  sound  cases  that  closely  relate  to  course  topics  is 
important  for  student  engagement  in  learning. 

The  case-based  learning  approach  can  be  equally  effective  in  online  as  it  is  in  a traditional  face-to-face 
classroom  settings  (Cheney,  2000,  as  cited  in  Rouke  & Anderson,  2002).  In  fact,  it  is  a tradition  that  case  studies 
have  been  conducted  in  individual  learning  modes,  not  collaborative  ones.  However,  Learning  is  an  engaging 
process  under  which  students  build  knowledge  through  social  interactions  and  collaboration  (Liu,  Lee,  Bonk, 
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Magjuka,  & Liu,  in  press).  Thus,  group  discussions  and  collaborations  are  the  most  important  parts  of  online  case- 
based  learning,  according  to  Rourke  and  Anderson  (2002). 

Findings  and  implications 

This  study  is  conducted  in  an  accredited  online  MBA  program  at  a top  ranked  business  school  in  a large 
Midwestern  university.  The  program  has  grown  to  include  hundreds  of  students  in  just  a few  years  and  was  reviewed 
as  a representative  program  to  examine  issues  of  online  case-based  learning.  Data  collected  in  this  study  included 
content  analyses  of  course  documents  and  class  assignments,  surveys  of  students,  and  interviews  of  faculty  members 
and  students. 

The  study  findings  confirm  that  case -based  learning  has  been  one  of  the  widely  used  instructional  methods 
in  the  online  MBA  program.  From  both  students  and  instructors’  perspectives,  the  attainment  of  the  equivalent 
quality  of  online  courses  with  their  counterparts  in  face-to-face  settings  seemed  to  be  an  underlying  goal,  and  at 
times,  a mandate,  for  delivering  online  courses.  While  this  creates  a positive  culture  in  this  program  for  achieving 
high  standards  of  online  education,  in  certain  ways  the  online  instructors  tended  to  replicate  the  classroom-based 
learning  as  much  as  possible  in  the  online  environment.  Thus,  many  instructors  retained  the  same  course  packet 
formats  for  teaching  business  content  in  online  courses.  Lengthy,  heavy  text-based  cases  still  were  the  dominant 
format  in  many  of  courses. 

Other  types  of  case  learning  such  as  personal  reflections,  students’  case  generation  and  evaluation  based  on 
personal  job  settings,  online  expert  commentaries  or  evaluations,  and  role  assignment  in  case  discussion  can  be 
considered  (Bonk  & Dennen,  2003;  Bonk,  Flara,  Dennen,  Malikowski,  & Supplee,  2000).  Such  studies  take 
advantage  of  the  diversified  adult  learners’  backgrounds  and  the  capabilities  of  technology  to  break  the  temporal  and 
physical  barriers.  Flowever,  these  activities  were  not  used  much  in  the  online  MBA  program.  In  stead,  simulation 
cases  were  used  in  several  courses  and  some  innovative  use  of  technology  (e.g.,  Breeze  chat  tool.  Case  Builder  tool, 
etc.)  for  online  case -based  learning  was  applied  to  ensure  the  quality  of  case  studies.  Interestingly,  several  pilot 
instructors  developed  ‘story -based  scenario’  and  the  scenarios  of  these  cases  were  embedded  in  an  online 
environment  with  threaded  discussions,  polling,  and  online  resource  exploration.  As  a result,  these  cases  proved  to 
be  more  engaging  and  appealing  from  the  students’  perspectives. 

Also,  the  study  indicates  that  there  were  several  factors  that  affect  instructors  in  transforming  the  format  of 
text-formatted  cases.  For  instance,  there  might  be  financial  constraints,  copyright  issues,  long  development  time, 
pedagogical  concerns,  and  heavy  uncertainty  about  the  educational  benefit  and  cost  ratio  of  case  transformation. 
These  concerns  and  issues  can  interfere  with  diverse  formats  of  delivery  such  as  multimedia  based  case  presentation 
for  dual  coding  effects  of  learning. 

On  the  other  hand,  the  instructors  extensively  vary  in  terms  of  choosing  individual  work  or  group  work  or 
some  combination  of  both  for  students  working  on  case  analysis.  In  quantitatively-geared  courses,  the  instructors 
tended  to  emphasize  individual  processing  skills  on  the  case  analysis.  In  more  judgmental  courses,  they  were  more 
focused  on  the  group  work  and  extensive  case  discussions.  In  general,  students  considered  group  work  is  a better 
approach  for  complex  case  problems.  However,  due  to  their  busy  schedule  and  locations  in  different  time  zones,  the 
difficulties  in  coordination  tended  to  be  a frustrating  experience  for  students  to  work  in  teams.  The  free-riding  issues 
also  affected  students’  satisfaction  in  teamwork  associated  with  online  case -based  learning. 

The  MBA  courses  varied  in  terms  of  the  degree  of  instructional  support  and  guidance  for  effective  case- 
based  learning  activities.  Some  instructors  took  advantage  of  the  courseware  tools  and  technological  features.  For 
instance,  the  course  announcement  in  the  course  management  systems  were  the  most  frequently  used  tool  for 
presenting  hints,  guidelines,  and  thinking  sheets  that  can  scaffold  students’  online  case -based  learning.  However, 
about  one-third  of  the  instructors  actually  moderate  discussion  forums  to  redirect  and  augment  students’  arguments. 

Asynchronous  discussion  forums  are  used  in  almost  all  the  courses  in  case-based  discussions  at  either  class- 
level  or  group-level.  While  most  instructors  tended  to  prefer  to  use  this  medium  as  tools  for  facilitating  in-depth 
reflection  on  cases  and  high  quality  discussions,  students  tended  to  be  frustrated  by  the  impersonal  feature  and 
delayed  exchange  of  information.  When  group  conducted  case  analyses  and  were  able  to  select  their  communication 
tools,  teleconferencing  tools  were  regarded  as  efficient  and  effective  tools  for  brainstorming  and  decision  making  in 
case -based  discussions.  In  the  meantime,  asynchronous  discussion  forums  were  mostly  used  to  upload  and  download 
deliverables  and  share  case  information.  Chat  rooms  were  not  used  effectively  in  facilitating  case-based  learning 
activities.  Part  of  the  reason  for  limited  use  of  real-time  chat  tools  were  the  low  comfort  level  of  instructors  with 
such  tools  and  the  constraints  of  chat  systems.  Simply  put,  it  is  not  easy  to  stimulate  in-depth  thinking  and  quality 
case  analyses  or  discussions  in  a synchronous  chat  environment.  Among  the  key  difficulties  was  scheduling  time  for 
participants  to  attend  a synchronous  chat  session.  Consequently,  email  is  used  by  instructors  to  post  instructions  and 
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specifications  of  the  case  analysis.  From  the  students’  side,  the  email  is  mainly  used  for  clarifying 
administrative  issues. 


Conclusion 

This  study  indicates  that  the  case -based  learning  method  is  a key  approach  in  the  online  MBA  program  just 
as  in  traditional  MBA  courses.  The  results  of  this  study  provide  instructors  and  educational  practitioners  with  some 
implications  for  instructors  around  the  globe  who  plan  to  implement  case-based  learning  activities.  Clearly,  those  in 
other  professional  schools  will  be  interested  in  practical  guidance  and  support  on  how  to  effectively  make  a 
successful  transition  of  the  traditional  case  study  method  to  the  online  case-based  learning  approach,  and  on  how  to 
embed  pedagogical  strategies  of  case -based  learning  in  their  online  courses. 

This  study  provides  a starting  point  for  understanding  the  current  status  of  case -based  learning  in  an  online 
MBA  program.  Further,  a systematic  research  agenda  need  to  be  established  in  the  areas  of  virtual  group  work, 
online  moderating,  online  mentoring,  online  performance  and  information  support,  and  tools  to  support  case-based 
learning  and  student  interactions. 

One  issue  centers  an  interest  in  socio-cultural  aspects  of  online  case -based  learning.  They  may  want  to 
know  how  to  design  effective  group  work  to  solve  online  cases,  how  to  embed  dilemmas  in  cases  that  support  social 
interactions  and  dialogue,  and  how  to  support  instructor  moderation  in  these  environments.  Further,  instructors  and 
support  staff  should  realize  that  the  success  of  online  business  courses  depends  on  the  appropriate  use  of  pedagogy 
into  business  contexts,  not  on  the  introduction  of  advanced  technology  itself.  Finally,  policy  makers,  foundations, 
and  government  agencies  should  be  aware  of  the  findings  here  and  target  funding  for  the  more  powerful  learning 
approaches  and  ideas. 
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Abstract 

The  purpose  of  this  project  was  to  design  a professional  development  program  to  support  teachers’ 
effective  use  of  web-enhanced  Problem-based  Learning  in  middle  school  science  classrooms. 
Implications  for  program  design  were  derived  from  empirical  research,  needs  of  practicing 
teachers,  and  practical  guidelines  from  teacher  professional  development  literature.  Synthesis  of 
our  research  results  suggested  the  following  key  design  features:  1)  model  PBL  to  teach  PBL,  2) 
balance  technology,  pedagogy,  and  the  PBL  method,  3)  enable  the  teachers  to  bridge  theory  and 
practice,  4)  enrich  the  experience  within  a Community  of  Practice,  5)  use  optimal  blending  of 
online  and  offline  learning  environments. 


Problem  Statement 

Problem-solving  skills  have  increasingly  been  recognized  as  desirable  skills  to  be  developed  in  Pre-K-12  education. 
According  to  Mayer  (1989),  “problem-solving  skills  should  be  learned  within  the  context  of  realistic  problem- 
solving situations.  Instead  of  using  drill  and  practice  on  component  skills  in  isolation  - as  suggested  by  the  skill- 
based  approach  - a meta-skill  based  approach  suggests  modeling  of  how  and  when  to  use  strategies  in  realistic 
academic  tasks”  (p.  53).  Problem-based  learning  is  one  pedagogical  strategy  that  helps  students  develop  deep 
understanding  within  a knowledge  domain  and  develop  effective  problem  solving  skills  especially  in  the  sciences. 
PBL  does  this  by  engaging  learners  in  the  learning  process  and  in  activities  to  solve  real  world,  authentic  problems 
(1996;  Hmelo  & Evensen,  2000;  Taconis  et  al.,  2001;  Voss,  1989). 

Although  many  studies  indicate  positive  impacts  of  PBL  on  student  learning,  as  a promising  pedagogical  strategy, 
this  practice  is  not  often  found  in  the  classroom  as  many  researchers  had  expected.  This  is  partly  because  PBL  is 
very  different  from  traditional  ways  of  teaching  such  as  teacher-centered  lectures  (Smith  et  al.,  1995).  As  a result 
PBL  may  not  be  easily  adopted  in  teaching  practice  or  adopted  in  ways  that  are  truly  PBL.  This  phenomenon  may 
also  be  explained  by  the  fact  that  many  teachers  may  not  have  had  enough  opportunities  to  develop  their 
understanding  and  skills  about  PBL  itself.  Considering  that  "the  ability  of  teachers  during  the  small  group  learning 
process  is  the  major  determinant  of  the  quality  and  the  success  of  PBL”  (Barrows,  1992),  it  appears  important  to 
foster  that  process  by  providing  teachers  with  specific  information  or  guidance  about  how  to  flexibly  use  or 
incorporate  a variety  of  instructional  approaches  to  PBL.  Current  guidance  for  PBL  implementation  still  is 
theoretically  oriented,  far  from  the  practitioners’  reality.  This  project,  therefore,  focused  on  designing  a framework 
for  professional  development  about  Web-Enhanced  PBL  that  is  scalable  and  replicable,  and  one  that  creates  a link 
between  theory  and  practice  in  using  PBL. 


Purpose  Statement 

The  goal  of  this  project  was  to  design  a professional  development  program  for  middle  school  science  teachers  who 
are  willing  to  use  PBL  more  effectively  in  their  classroom.  Implications  for  program  design  were  derived  from 
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empirical  research,  needs  of  practicing  teachers,  and  practical  guidelines  from  teacher  professional 
development  literature. 


Context  of  the  Study 

The  professional  development  program  is  designed  to  facilitate  the  teachers’  effective  use  of  Web-enhanced  PBL 
program,  using  an  existing  program  named  KaAMS  (Kids  as  Airborne  Mission  Scientists.  See  the  website  at 
http://ide.psu.edu/kaams/kaams).  The  KaAMS  web  site  provides  PBL-based  lesson  plans  and  teaching  resources  fro 
middle  school  science.  KaAMS,  as  a NASA  funded  project,  was  created  to  help  teachers  inspire  middle  school 
students  to  learn  science  by  integrating  a variety  of  web  resources  in  a flexible  and  authentic  way. 

Approach  to  the  Program  Design 

Literature  Review  Results  on  Teachers  and  Teaching  Practice 

The  design  of  the  teacher  professional  development  program  for  KaAMS  is  grounded  on  a literature  review  of 
teaching  practice.  From  the  literature,  several  teachers’  characteristics  were  predicted  to  influence  the  use  of  general 
teaching  strategies  including  problem-based  learning  (Grabowski  et  ah,  2000).  More  specifically,  the  researchers 
found  five  major  variables  related  to  the  technology  use  and  PBL  teaching  methods:  computer  and  Internet  skills, 
feeling  of  preparedness  to  use  the  Internet,  attitude  toward  the  use  of  web  resources,  pedagogical  beliefs,  and  science 
teaching  efficacy.  These  variables  were  believed  to  influence  science  teaching  practice  (Ching  et  al. , 2005)  . Thus, 
the  researchers  consider  those  factors  in  program  curriculum  design  so  that  the  terminal  goal  of  ‘supporting  teachers 
to  use  PBL  more  effectively  in  their  classroom  activities’  can  be  achieved. 

Empirical  Study  Results  on  Teacher  and  Teaching  Practice 

The  researchers  conducted  a study  examining  the  interrelationships  between  teacher  characteristics  and  teaching 
practice  using  several  predictor  variables  as  identified  from  the  literature.  (See  Appendix  A).  The  major  purpose  of 
this  study  was  to  get  closer  to  the  teachers  with  questions  about  who  they  are  and  what  they  do  in  their  classroom. 
Most  of  all,  the  empirical  data  from  this  study  served  as  direct  input  into  the  design  of  the  teacher  professional 
development  program.  To  verify  the  theoretical  model  which  emerged  from  the  literature  review,  twenty  seven 
middle  school  science  teachers  ranging  from  fifth  grade  to  eighth  grade  participated  in  the  study.  (Lim  et  al,  2006). 

The  results  suggest  some  interesting,  yet  meaningful,  implications  for  designing  a teacher  professional  development 
program.  First,  teachers’  computer  and  internet  skills  appear  to  be  highly  influential  to  teaching  practice.  For 
example,  a teacher  who  has  higher  computer  skills  is  likely  to  adopt  more  varied  teaching  strategies  such  as  PBL  or 
an  Internet-based  activity,  and  the  vice  versa.  Second,  teachers’  science  teaching  efficacy  was  negatively  correlated 
attitude  toward  web  resources.  In  other  words,  a teacher  who  has  higher  science  teaching  efficacy  shows  negative 
attitude  toward  Internet  resources.  This  can  be  interpreted  as  a teacher  with  high  efficacy  may  not  be  dependent  on 
online  resources.  A third,  pedagogical  belief  was  a critical  variable  in  predicting  teacher’s  teaching  practice, 
especially  PBL  use.  It  is  notable  that  there  was  a significant  correlation  between  computer  and  Internet  skills  and 
pedagogical  beliefs,  which  means  there  is  a high  probability  that  teachers  who  have  higher  level  of  computer  and 
Internet  skills  might  believe  in  the  constructivist  approach. 

Implications  for  design: 

■ Improve  computer  and  Internet  skill 

■ Provide  a chance  to  reflect  on  the  usefulness  of  web  resources,  with  the  consideration  of  teachers’  level  of 
teaching  efficacy. 

■ Develop  a positive  attitude  toward  the  pedagogy  of  Constructivism. 

Teacher  Interview  Results  on  Web-enhance  PBL  Use 

Several  middle  school  science  teachers  were  directly  involved  in  the  process  of  program  design.  In  January  2006,  as 
a part  of  a pilot  study,  two  science  teachers  in  Pennsylvania  implemented  the  KaAMS  PBL  lesson  plan  in  their 
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classroom  to  provide  rich  feedback  on  KaAMS  application,  while  another  two  teachers  reviewed  the  KaAMS 
resources  to  provide  suggestions  and  comments  on  KaAMS  program.  The  data  from  these  teachers,  as  one  of  the 
most  important  data  sources,  answer  the  questions  such  as;  what  kind  of  difficulties  do  teachers  encounter  when 
using  the  KaAMS  PBL  resources?  or  do  they  need  more  information,  resources,  or  support  during  the  PBL 
implementation?  Ongoing  reading  of  the  qualitative  data  was  complemented  with  the  constant  comparison  method 
of  data  analysis  provided  by  Glaser  and  Strauss  (1967).  The  collective  experience  of  all  four  teachers  formed  the 
basis  for  the  three  themes  that  emerged:  frustration,  teacher  as  a facilitator,  and  potential.  Throughout  the  course  of 
the  research  study  other  issues  and  concerns  came  to  light,  but  it  was  these  recurrent  three  themes  that  remained 
constant  throughout. 

At  the  beginning  of  implementation,  the  teachers  felt  frustrated  when  trying  to  adopt  this  new  teaching  practice  and 
activity  recommended  in  web-enhanced  PBL.  Although,  the  web-enhanced  PBL  program  provided  a detailed 
teacher  manual,  the  teachers  perceived  this  new  teaching  approach  as  different  from  their  own  teaching  practice.  A 
lack  of  resources,  especially  hardware,  was  another  barrier  to  implementing  the  web-enhanced  PBL  lesson  plans. 
Time  constraint  was  a key  reason  given  by  the  teachers  as  to  why  they  were  hesitant  about  using  this  new 
curriculum.  Finally,  the  State  test  was  another  big  cause  of  their  frustration  because  they  felt  these  regulations 
prevented  them  from  implementing  new  pedagogical  approaches.  Although  they  were  frustrated  with  this  new 
approach,  they  seemed  gradually  to  integrate  KaAMS  into  their  classrooms.  Interestingly,  all  of  four  teachers’ 
pedagogical  beliefs  showed  that  they  preferred  to  use  a constructivist  approach  in  their  science  classroom  that 
focused  on  authentic  problem  solving,  collaborative  learning,  and  promoting  students’  thinking.  They  indicated  that 
this  new  teaching  approach,  also,  had  affected  their  perceptions  on  their  role  in  teaching  science— changing  from  a 
didactic  teacher  to  that  of  a facilitator.  During  and  after  participating  in  this  project,  the  teachers  believed  that  web- 
enhanced  learning  environments  make  students’  learning  more  dynamic  and  active,  and  motivate  students  to  actively 
engage  in  their  learning  process.  Teachers  perceived,  also,  that  this  new  practice  encouraged  their  students  to 
actively  engage  in  communication  or  interaction  between  students  and  teachers. 

Implications  for  design: 

■ Minimize  frustration,  maximize  success,  and  demonstrate  the  flexibility  of  the  approach:  the  teachers  felt 
frustrated  when  trying  to  adopt  this  new  teaching  practice  since  they  don’t  have  enough  time  and  the 
pedagogical  approach  is  different  from  what  they  are  used  to. 

■ Convince  teachers  that  this  approach  is  effective  from  the  beginning:  All  the  teachers  believed  that  this 
practice  was  effective  especially  in  science  teaching  after  implementing  this  approach.  So,  right  from  the 
beginning,  it  is  important  to  convince  teachers  of  the  effectiveness  of  this  approach. 

Practical  Considerations  in  Teacher  Professional  Development 

Time,  accessibility  and  financing  are  critical  to  teachers’  choice  of  and  participation  in  the  professional  development 
activities  (T.M.  Duffy  et  al.,  2006).  Given  the  busy  schedules  and  workload  of  middle  school  teachers,  teachers 
typically  only  participate  in  a program  that  lasts  less  than  8 hours  (Parsad  et  al,  2001).  While  short  programs  meet 
teachers’  practical  concerns,  previous  studies  also  showed  that  the  effects  of  one-shot  programs  are  limited.  They 
may  promote  teachers’  awareness  or  interests  in  the  new  ideas  or  skills  (Boyle  et  al,  2005),  but  are  insufficient  in 
terms  of  preparing  them  for  adopting  new  teaching  methods,  or  integrating  new  skills.  By  reviewing  a large  body  of 
literature,  Boyle  et  al.  (2005)  suggested  that  professional  development  activities  over  longer  periods  of  time  have 
stronger  sustaining  influence  on  teaching  practice  because  they  offer  teachers  opportunities  to  go  through  an 
iterative  process  of  practice  and  reflection  on  what  they  have  learned.  The  findings  of  their  survey  further  revealed 
that  longer  term  professional  development  activities  led  to  changes  in  one  or  more  aspects  of  teachers’  teaching 
practice.  In  short,  to  design  a professional  development  program,  the  challenge  lies  on  how  to  find  the  balance 
between  practicality  and  learning  effectiveness. 

With  the  advancement  of  the  internet,  online  delivery  of  professional  development  programs  seems  to  be  a solution 
to  overcome  teachers’  practical  concerns  about  professional  development.  However,  professional  development 
programs  that  are  fully  delivered  online  are  at  the  expense  of  face-to-face  interaction.  Programs  that  need  to  affect  a 
change,  especially  related  to  developing  skills  of  new  teaching  practice  with  technology,  would  benefit  from  face-to- 
face  interaction.  A face-to-face  program  accompanied  by  online  supporting  (professional  or  peer  supports)  systems, 
namely  a blended  method,  should  be  a promising  solution.  Learning  in  a blended  environment  allows  the 
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convenience  brought  by  an  online  environment  while  not  sacrificing  the  face-to-face  social  interactions  (Osguthorpe 
& Graham,  2003). 

Implications  for  design: 

■ Prolong  the  professional  development  to  improve  its  effectiveness 

■ Provide  sustainable  online  support  systems  to  enhance  face-to-face  interactions 

Conceptual  Framework  for  the  Design 

These  implications  from  research  and  practice  informed  the  development  of  a conceptual  framework  for  the 
professional  development  program  whose  purpose  was  to  facilitate  middle  school  science  teachers  becoming  experts 
in  implementing  web-enhanced  PBL  in  their  classroom. 

Apply  PBL  to  Teach  PBL 

By  situating  teachers  in  a PBL  learning  environment  themselves,  this  professional  development  program  will  help 
teachers  develop  an  understanding  of  PBL  from  a cognitive,  psychomotor  and  affective  perspective.  By  following 
the  PBL  process  to  learn  PBL,  the  teachers,  first,  will  need  to  assess  their  existing  knowledge  of  technology, 
pedagogical  beliefs,  and  PBL  through  self-diagnosis.  Then  they  will  explore  and  develop  a deep  and  contextualized 
understanding  of  Web-enhanced  PBL  ( cognitive ).  While  practicing  the  PBL  strategy  successfully 
( psychomotor/skill ) during  the  program,  teachers  are  likely  to  feel  the  benefits  and  build  confidence  around  adopting 
this  strategy  in  their  classrooms  ( affective ).  This  design  addresses  all  three  fundamental  elements  (i.e.,  cognitive, 
affective  and  psychomotor)  of  an  attitude  (Kamradt  & Kamradt,  1999).  It  is  our  expectation  that  teachers  will  form  a 
positive  attitude  toward  the  adoption  of  Web-enhanced  PBL  after  participating  in  this  program. 

The  design  of  this  professional  development  program  follows  a PBL  process  synthesized  from  different  PBL  models 
(Hmelo-Silver,  2004;  Torp  & Sage,  1998).  We  start  from  presenting  the  problem  scenario  to  set  the  context  for 
learning,  and  then  we  help  teachers  to  self-diagnose  what  they  know  and  what  they  do  not  know  about  the  problem. 
Through  collecting  information  about  the  unknown,  teachers  learn  deeply  about  the  problem  and  form  their  own 
solutions  to  the  problem.  In  the  last  stage,  teachers  will  present  their  solutions  to  the  public  and  share  their  learning 
processes. 

Balance  Technology,  Pedagogy,  and  PBL  Method 

The  literature  review  and  empirical  research  results  revealed  that  the  computer  and  internet  skill  and  more 
constructivist  pedagogical  beliefs  are  both  critical  to  predicting  the  use  of  PBL  in  the  classroom.  Accordingly,  the 
researchers  have  included  technology  skills  as  well  as  pedagogical  understanding  in  the  professional  development 
program.  Without  these  two  components,  the  terminal  goal  of  helping  teachers  integrate  web-enhance  PBL  would 
not  be  accomplished.  Figure  1 and  Table  1 depict  how  technology,  pedagogy,  and  PBL  method  are  balanced  as  three 
major  themes  of  this  program. 

Provide  a Bridge  between  Theory  and  Classroom  Practice 

Knowledge  about  theories  is  not  sufficient  to  become  expert  PBL  teachers.  A link  to  success  is  to  provide 
opportunities  for  teachers  to  ‘practice  in  their  own  classroom’  so  that  they  are  able  to  transfer  and  sustain  what  they 
learned.  From  this  point  of  view,  the  program  is  designed  to  provide  a link  between  theory  and  “classroom  practice” 
in  each  theme.  More  specifically,  every  theme  starts  from  the  basic  concepts  and  ideas  related  to  the  theme  taught 
using  a PBL  approach  as  described  above,  followed  by  in-depth  discussion  about  application  and  integration.  After 
finishing  each  offline  session,  teachers  return  to  their  classroom  and  apply  what  has  been  discussed  during  the 
program.  The  reflective  action  plan  that  is  developed  at  the  end  of  each  theme  connects  the  teacher  professional 
development  program  and  their  reality,  the  classroom. 

Enrich  the  Experience  within  a Community  of  Practice 

Considering  that  one  of  the  implications  for  design  is  to  prolong  the  professional  development  experience,  providing 
a community  of  practice  is  quite  important.  Researchers  interpret  the  word  ‘prolong’  not  only  in  the  context  of  the 
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length  of  hours  but  also  of  the  depth  of  engagement.  Within  a community  of  practice,  teachers  learn,  share,  and 
reflect,  which  enables  them  to  solve  their  own  problems  in  their  own  real  context  and  to  build  a case  library 
throughout  the  iterative  process  of  learning.  The  Community  of  Practice  component  is  designed  based  on  the  theory 
of  Case -based  Reasoning  by  Roger  Shank,  and  Situated  Cognition  by  Lave  and  Wenger.  The  application  of  PBL  and 
integrating  online  space  are  the  enablers  of  building  a community  of  practice  among  the  teachers. 

Deliver  Via  Optimal  Blending  of  Online  and  Offline 

According  to  the  design  implications  from  a practical  perspective,  blending  of  online  and  offline  is  strongly 
recommended  to  pursue  both  efficiency  and  social  interaction.  The  design  of  this  professional  development  program 
maximizes  the  strength  of  each  delivery  method  (See  Table  1).  First,  teachers  improve  their  computer  and  internet 
skill  over  the  internet  to  equalize  entry  behaviors,  while  starting  to  share  information  with  and  about  each  other.  The 
basic  concept  of  mastery  learning  is  applied  at  this  point  of  warming  up.  Second,  all  of  the  teachers  get  together 
face-to-face  for  a day  (per  each  theme)  to  work  collaboratively.  This  offline  activity  is  expected  to  increase  the 
quality  of  ongoing  interaction  with  the  group  work  increasing  the  opportunities  to  getting  to  know  each  other. 
Teachers,  then,  return  to  their  school,  but  with  the  expectation  that  learning  will  be  ongoing.  Teachers  carry  out  their 
action  plans  while  communicating  with  experts  and  peers.  The  combination  of  1 day  offline  and  1 week  online  is 
repeated  for  each  theme  to  maximize  the  effectiveness  and  minimize  the  cost  of  time  and  travel. 
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Figure  1.  Conceptual  Structure:  Learning  to  Use  Web-Enhanced  PBL 
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Based  on  the  conceptual  framework,  details  of  the  professional  development  program  to  facilitate  teacher’s  ability  to  implement  web-enhanced  PBL  are  shown 
in  Table  1. 
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Conclusion 


The  purpose  of  this  project  was  to  design  a professional  development  program  to  support  teachers’  effective  use  of 
web-enhanced  Problem-based  Learning  in  middle  school  science  classrooms.  Design  implications  for  the  program 
came  from  several  years  of  empirical  research,  listening  to  practicing  teachers  who  used  KaAMS  lesson  plans,  and 
reviewing  practical  guidelines  from  teacher  professional  development  literature.  Synthesis  of  our  research  results 
suggested  the  following  key  design  features:  1)  model  PBL  to  teach  PBL,  2)  balance  technology,  pedagogy,  and  the 
PBL  method,  3)  enable  the  teachers  to  bridge  theory  and  practice,  4)  enrich  the  experience  within  a Community  of 
Practice,  and  5)  use  optimal  blending  of  online  and  offline  learning  environments.  Blended  with  the  practical  and 
empirical  results  is  our  own  understanding  and  beliefs  about  learning  as  informed  by  learning  and  instructional 
theory.  As  shown  in  the  Conceptual  Framework,  mastery  learning,  case  based  reasoning,  attitudinal  change, 
problem  based  learning  and  situated  cognition  were  prominent  in  helping  to  develop  the  details  of  the  design.  The 
resulting  professional  development  program,  therefore,  is  more  than  a workshop  for  teachers.  It  alternates  between 
carefully  selected,  appropriate  face-to-face,  offline  and  online  expert  and  peer  supported  experiences  that  maximize 
learning,  efficiency  and  transfer  of  theory  into  classroom  practice. 


*This  study  is  supported  by  funding  from  the  Penn  State  University  Thematic  Initiation  Fund. 
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Appendix  A:  Conceptual  Framework  of  Correlational  Results  (Lim  et  al.,  2006) 
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The  Need  for  a Research  Agenda 

Video  games  have  exceeded  U.S.  box-office  in  the  annual  sales  (Tran,  2002)  and  emerged  as  the  most 
frequently  used  interactive  media  among  children  (Kirriemuir  & McFarlane,  2003).  Game-based  learning  advocates 
(Gee,  2003;  Prensky,  2001a)  argue  that  this  powerful  medium  can  and  should  be  utilized  for  laudable  educational 
outcomes.  However,  although  teachers  and  administrators  acknowledge  the  positive  effect  of  computer  games  on 
students’  collaboration,  discussion,  and  thinking  skills,  they  are  not  convinced  that  games  can  be  used  as  part  of  the 
school  curriculum  (McFarlane,  Sparrowhawk,  & Heald,  2002).  Instead,  the  type  of  educational  games  that  schools 
are  familiar  with  is  edutainment,  which  usually  focuses  on  the  motivational  effects  of  games  typically  employing 
game-like  drill  and  practice  activities  to  achieve  lower  level  learning  goals.  Many  of  the  edutainment  endeavors, 
which  use  the  entertaining  elements  in  video  games  to  make  learning  fun,  failed  to  produce  anything  either 
educational  or  entertaining  (Kirriemuir  & McFarlane,  2003;  Okan,  2003;  Prensky,  2001b).  Additionally,  researchers 
have  limited  experience  designing  or  implementing  effective  game-based  learning,  especially  with  educational 
computer  games.  Although  there  exists  40-years  of  research  on  educational  games,  the  research  community  is  small 
and  many  research  issues  remain  unresolved. 

What  research  should  be  conducted  to  build  a knowledge  base  that  supports  the  design  and  implementation 
of  educational  computer  games?  Hannafin  and  Kim’s  (2003)  critique  of  web-based  teaching  research  can  inform 
researchers  interested  in  educational  computer  games.  They  criticized  web-based  teaching  research  as  being  too 
diffuse  and  contradictory;  many  studies  reexamined  design  issues  that  have  already  been  investigated  in  similar 
media  in  the  past.  They  argue  that  web-based  teaching  research  should  explore  whether  existing  theory  and  research 
provides  guidance  to  web-based  learning,  which  principles  are  relevant,  and  which  inquires  should  be  conducted  to 
build  a knowledge  base  for  web-based  learning.  They  advocate  that  researchers  should  examine  the  unique  attributes 
of  the  web  to  develop  teaching  and  learning  processes  that  are  philosophically  different  from  traditional  approaches. 
We  believe  this  advice  is  valuable  for  researchers  interested  in  educational  computer  games  as  well.  As  a research 
community,  we  need  to  examine  relevant  learning  and  instructional  theories  to  identify  guidance  for  designing  and 
implementing  educational  computer  games.  We  should  also  review  previous  research  on  educational  games,  whether 
they  are  electronic  or  non-electronic,  to  determine  what  has  been  studied  and  what  issues  remain  to  be  addressed.  In 
addition,  rather  than  focusing  on  all  design  issues  related  to  educational  computer  games,  it  is  probably  more 
productive  to  examine  educational  affordances  unique  to  computer  games.  It  is  in  this  spirit  that  we  initiated  our 
own  research  on  educational  games  and  developed  a research  agenda  to  guide  our  future  work. 

To  provide  guidance  for  our  own  development  and  implementation  of  an  educational  computer  game,  we 
examined  current  learning  and  instructional  theories  and  the  affordances  of  video  games  to  identify  design  guidance 
that  leverages  the  strengths  of  video  games  and  various  theories  such  as  problem-based  learning  (Barrows,  1996), 
situated  cognition  (Brown,  Collins,  & Duguid,  1989),  cognitive  apprenticeship  (Collins,  Brown,  & Holum,  1991),  to 
name  just  a few.  We  distilled  several  main  principles  for  designing  our  own  educational  computer  game  (Ma, 
Williams,  Richard,  Prejean,  & Liu,  2006):  1).  situating  problems  in  a rich  context  in  order  to  engage  students  in 
scientific  inquiry  that  reflects  the  way  experts  work;  2).  present  problems  in  all  their  complexity  and  offer  tools, 
resources,  and  scaffolds  to  make  complexity  manageable;  3).  provide  learners  with  increased  power  of  agency;  4). 
provide  opportunities  for  authentic  assessment  of  student  performance.  How  do  these  guidelines  work  in  educational 
computer  games?  What  new  strategies  or  new  theories  should  be  developed  and  tested  to  guide  educational 
computer  games?  This  paper  presents  a research  agenda  that  explores  these  issues. 
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Potential  Research  Areas 


The  first  four  research  areas  listed  below  are  related  to  the  four  design  guidelines  we  generated  from  the 
literature.  The  rest  of  the  research  areas  concern  other  important  factors  impacting  the  use  of  educational  computer 
games. 

Problem  Presentation 

Many  contemporary  theories  on  learning  environments  are  built  around  a central  problem  in  a rich  context 
(Barrows,  1996;  Hannafm,  Hill,  & McCarthy,  2000;  Jonassen,  1999;  Schank,  Fano,  Bell,  & Jona,  1994;  Spiro  & 
Jehng,  1990).  Modern  video  games  provide  opportunities  for  immersive  and  authentic  representations  of  problems 
with  the  use  of  rich  narratives,  3-D  environments,  real-time  cinematics,  as  well  as  stirring  audio  effects  and  music. 
What  media  components  should  be  used  to  present  the  problem  context  in  a certain  game?  How  do  these  different 
media  components  in  problem  presentation  impact  learning?  Pre-rendered  movies  or  real-time  cinematics  are  two 
common  strategies  used  to  present  the  context  in  commercial  computer  games.  Is  one  strategy  more  appropriate  than 
the  other  for  presenting  certain  problems  in  the  educational  computer  game? 

In  some  commercial  computer  games,  problems  or  tasks  are  presented  to  the  players  when  they  approach  a 
non-player  character  (NPC)  to  receive  a quest  or  quests.  These  NPCs  may  serve  as  pedagogical  agents  (Baylor  & 
Ryu,  2003)  in  educational  computer  games.  What  roles  can  a pedagogical  agent  play  in  educational  computer 
games?  In  addition  to  using  a NPC  as  a pedagogical  agent,  can  a teacher  play  as  an  agent  when  the  need  arises? 
Could  the  facilitation  and  scaffolding  role  be  shared  between  the  agents  played  by  the  teacher  and  NPC?  What 
scaffolding  tasks  are  more  appropriate  for  NPC?  What  tasks  are  appropriate  for  teachers?  What  impact  does  a 
pedagogical  agent  have  on  learner  performance  and  motivation  in  educational  computer  games? 

In  computer  games,  problems  are  organized  either  in  a structured  “quest”  or  an  unstructured  “sandbox” 
format.  What  format  (“quest”  or  a “sandbox”)  is  more  appropriate  for  a particular  type  of  problems,  learning 
outcomes,  or  content  areas  in  educational  computer  games? 

Facilitation  Strategies 

In  the  learning  and  instructional  theories  cited  in  this  paper  (Barrows,  1996;  Hannafm  et  ah,  2000; 

Jonassen,  1999;  Schank  et  ah,  1994;  Spiro  & Jehng,  1990),  the  role  of  the  instructor  shifts  from  the  “saint  on  the 
stage”  to  the  “guide  on  the  side.”  Facilitation  strategies  are  the  key  component  to  enable  learning  in  educational 
computer  games.  For  example,  Egenfeldt-nielsen  (2005)  found  that  teachers  play  a critical  role  in  helping  students 
make  a connection  between  game  playing  and  scientific  concepts.  He  found  that  although  educational  computer 
games  provide  experiences  needed  for  students  to  construct  knowledge,  the  deep  understanding  of  the  scientific 
concepts  does  not  come  automatically.  Although  students  may  develop  spontaneous  concepts  themselves; 
developing  scientific  concepts  requires  the  facilitator  to  help  make  the  connection.  The  two  facilitators  in  his  studies 
were  overwhelmed  by  the  responsibilities  and  failed  to  deliver  the  assistance  needed  by  students.  This  study  reveals 
that  despite  the  fact  that  educational  computer  games  have  great  potential  to  benefit  education,  there  are  many 
challenges  for  facilitation. 

What  support  do  students  need  to  maximize  the  educational  benefit  of  computer  games?  What  facilitation 
strategies  should  be  provided  in  educational  computer  games?  Learning  and  instructional  theories  suggest  various 
facilitation  strategies  (Barrows,  1996;  Brown  et  al.,  1989;  Collins  et  ah,  1991;  Schank  et  ah,  1994;  Spiro  & Jehng, 
1990).  For  example,  cognitive  apprenticeship  recommends  strategies  such  as  modeling,  coaching,  scaffolding, 
articulation,  reflection  and  exploration  (Collins  et  ah,  1991).  How  are  these  strategies  used  in  educational  computer 
games?  How  can  support  and  scaffolding  be  built  into  the  game  to  alleviate  facilitators’  responsibilities?  How 
effective  are  these  strategies?  Moreno  & Mayer  (2005)  found  that  guidance  in  the  form  of  explanatory  feedback 
resulted  in  improved  student  learning  in  an  interactive  game,  but  reflection  in  the  form  of  asking  students  to  give 
explanations  for  their  answers  did  not  affect  learning.  New  findings  from  cognitive  and  learning  sciences  may  also 
suggest  strategies  for  use  in  educational  computer  games.  For  example,  researchers  who  study  analogical  encoding 
(i.e.  learning  by  comparing  two  cases)  found  that  teaching  students  to  compare  cases  side-by-side  improves 
knowledge  acquisition  and  transfer  (Kurtz,  Miao,  & Centner,  2001).  Analogical  encoding  aids  students  in  attending 
to  the  deep  underlying  structures  (i.e.  principles)  of  cases  thereby  reducing  potential  distraction  of  surface  features. 
Our  team  is  developing  an  anagogical  encoding  tool  in  an  educational  computer  game  to  scaffold  how  to  draw 
analog  from  two  cases  in  order  to  solve  new  problems  (Williams,  Feist,  & Ma,  October  2006). 
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Computer  games  have  unique  characteristics  that  may  afford  innovative  approaches  to  implementing 
facilitation  and  scaffolding  strategies.  For  example,  who  should  provide  modeling  in  educational  computer  games? 
Should  some  modeling  be  provided  in  the  game  by  a NPC  or  some  by  an  agent  played  by  the  teacher?  What 
modeling  tasks  are  appropriate  for  NPCs  and  what  are  appropriate  for  teachers?  Flow  do  these  strategies  impact 
learning?  Moreover,  many  games  have  Fleads  Up  display  (FIUD),  status  monitoring  tools,  resources,  community 
building  tools,  and  tutorials  (Pellegrin,  Ma,  Williams,  & Kunnel,  2006).  Flow  can  these  tools  add  to  or  enhance  the 
tools  and  resources  described  in  various  computer-based  learning  environments  (Flannafin  et  ah,  2000;  Jonassen, 
1999)? 

Narratives  are  a key  component  in  many  computer  games.  By  and  large,  the  conventions  of  western 
narrative  are  founded  on  aesthetic  principals  advanced  by  Aristotle,  who  argued  that,  at  its  best,  narrative  has  the 
capacity  to  be  both  entertaining  and  edifying.  Character  and  plot  are  two  key  elements  in  this  convention.  Characters 
act  to  pursue  the  object  of  their  desire  or  motivation.  In  the  course  of  their  pursuit  or  quest,  characters  encounter 
obstacles  or  problems  that  interfere  with  achieving  their  goal.  Characters  seek  solutions  to  problems  and  take  action 
to  overcome  obstacles.  This  narrative  structure  has  much  in  common  with  stories  and  cases,  key  elements  used  in 
learning  and  instructional  theories  such  as  learning  by  design  (Kolodner  et  ah,  2003),  goal-based  scenarios  (Schank 
et  ah,  1994),  and  analogical  encoding  (Kurtz  et  ah,  2001).  What  role  should  narrative  play  in  educational  computer 
games  in  light  of  these  theories?  For  example,  goal-based  scenarios  (Schank  et  ah,  1994)  emphasize  the  use  of 
stories  not  only  to  present  the  problem,  but  also  serve  as  tools  to  facilitate  reflection,  as  a resource  to  provide  advice, 
and  to  provide  feedback  to  show  consequences  of  actions. 

Formative  Assessment 

Formative  assessment  plays  a critical  role  in  effective  teaching  (Brandsford,  Pellegrino,  & Donovan,  1999). 
It  reduces  the  learning  time  needed,  better  motivates  students  (Corbett  and  Anderson  cited  in  Koedinger,  1998),  and 
leads  to  strong  learning  gains  (Black  & Wiliam,  1998).  It  is  believed  that  immediate  feedback  may  contribute  to  the 
experience  of  flow  (Csikszentmihalyi,  1991).  Despite  its  effectiveness  in  bringing  about  learning  gains,  formative 
assessment  is  lacking  in  schools  (Black  & Wiliam,  1998)  because  it  is  extremely  time  consuming  and  it  places  high 
demands  on  the  teacher. 

Video  games  offer  many  opportunities  to  monitor  and  assess  players’  performance  that  goes  beyond  those 
typically  found  in  traditional  learning  environments.  The  processing  speed  and  networking  of  modern  computers 
allow  players-as-characters  to  interact  with  their  simulated  environments  and  circumstances,  and  with  other  players, 
in  real  time.  Not  only  does  this  increase  the  plausibility  of  the  fiction  and  allow  the  players  to  immerse  themselves 
further  in  the  character  and  plot,  but  it  also  reinforces  the  connection  between  decision-making  and  outcome. 
Moreover,  interactions  among  multiple  players  provide  opportunities  for  peer  feedback,  collaborative  problem- 
solving and  community  building  (Kolo  & Baur,  2004;  McFarlane  et  al.,  2002).  In  addition,  video  games  allow 
players  to  retain  and  access  great  volumes  of  data,  either  through  quantitative  measurements  such  as  score -keeping, 
through  item  acquisitions,  increases  in  abilities  and  aptitudes,  verbal  record-keeping  devices,  and  mapping.  The 
persistence  of  data  is  a key  feature  of  the  medium,  allowing  players  to  assess  their  own  performances  against  that  of 
the  system  or  of  other  players.  This  feature  may  also  become  a critical  tool  for  performance  assessments  by  external 
observers  such  as  a teacher.  Video  game  techniques  and  technologies  may  provide  more  frequent  and  effective 
means  to  assess  students’  performance.  What  tools  can  be  built  into  educational  computer  games  to  conduct 
formative  assessment?  Computer-based  formative  assessment  tools  have  used  sophisticated  multiple  choice 
questions  (Flunt  & Pellegrino,  2002)  or  natural  language  processing  techniques  such  as  latent  semantic  analysis 
(Landauer  & Dumais,  1997)  to  assess  learning.  These  tools  can  be  integrated  into  educational  computer  games. 
Flowever,  current  technologies  may  not  have  the  intelligence  to  replace  teachers  for  all  formative  assessment  tasks 
required.  What  formative  assessment  should  be  conducted  by  teachers?  Flow  can  the  results  of  assessment  be  used  to 
customize  an  individual’s  learning  experience?  In  one  study,  Squire,  Giovanetto,  Devane,  and  Durga  (2005)  found 
that  a key  role  the  facilitators  played  was  to  constantly  monitor  and  assess  students’  game  playing.  When  the  game 
was  too  complex  for  students,  the  facilitators  had  to  create  simplified,  customized  game  scenarios  on  the  fly.  This 
type  of  assessment  and  facilitation  requires  high  facilitator  ratio,  which  might  not  be  feasible  in  many  situations. 
More  research  is  needed  in  this  area. 

Learner  Agency 

Learner  agency  is  a key  factor  that  may  contribute  to  intrinsic  motivation  (Malone,  1981)  and  flow 
(Csikszentmihalyi,  1991).  Video  games  provide  the  players  with  increased  power  of  agency.  Simulations  are  found 
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in  many  video  games.  They  enable  the  players  to  transcend  the  constraints  of  time  and  space  to  explore  the 
dynamics  of  a real  or  imaginary  system  (Alessi  & Trollip,  1991).  Players  manipulate  variables  and  test  hypotheses  to 
construct  mental  models  of  a domain  (Rieber,  1996).  Moreover,  in  video  games,  simulations  are  embedded  in 
interactive  storytelling  (Murray,  1997).  In  traditional  storytelling,  audiences  are  simply  spectators  to  the  drama, 
passively  observing  the  motives,  decision-making,  and  actions  of  characters  onstage.  In  video  games,  however,  the 
players  identify  fully  with  a character,  act  on  the  plot,  and  experience  the  cause-and-effect  of  their  actions. 
Additionally,  cutting-edge  AI  technologies  offer  players  a choice  of  roles  and  tasks,  and  challenge  them  with  tasks 
appropriate  for  their  ability  level.  This  allows  the  player  to  have  control  over  their  experience  and  to  engage  in 
problems  within  their  zone  of  proximal  development  (Vygotsky,  1978). 

Video  games  may  provide  players  with  increased  power  of  agency.  What  strategies  can  be  used  to  enhance 
a sense  of  agency  in  educational  computer  games?  How  do  these  strategies  impact  learning  and  motivation?  For 
example,  one  question  concerns  learner  agency  and  academic  achievement.  Players  have  preferences  in  terms  of  the 
types  of  activities  they  would  pursue  in  games  (Bartle,  2005).  Some  are  more  aspired  to  win  the  game;  some  enjoy 
more  to  explore  the  world;  some  spend  more  time  socializing;  and  others  focus  more  on  dominating  competitors.  In 
educational  computer  games,  how  to  help  children  with  different  preferences  to  achieve  academic  goals  but  still  give 
them  a sense  of  agency  by  allowing  them  options  and  freedom  to  pursue  their  interests? 

Settings  for  Educational  Game  Use 

Several  researchers  suggested  that  using  regular  school  hours  for  educational  games  might  be  a problem, 
because  of  the  time  schedule  and  the  existing  script  for  work  and  roles  that  occurs  during  regular  school  hours 
(Egenfeldt-nielsen,  2005;  Saegesser,  1981).  Computer  games  may  last  for  hours,  yet  regular  school  hours  are 
divided  into  small  class  periods.  It  is  difficult  to  confine  game  play  within  a short  period  of  time.  Moreover, 
classroom  teaching  is  usually  teacher-centered  and  schooling  is  considered  work,  whereas  games  are  more  player  or 
learner-centered  and  games  are  perceived  as  play.  Research  on  the  use  of  computer  games  has  taken  place  in  regular 
school  hours  in  the  classroom  setting  (Barab,  Thomas,  Dodge,  Carteaux,  & Tuzun,  2005;  Egenfeldt-nielsen,  2005; 
Ketelhut,  Clarke,  Dede,  Nelson,  & Bowman,  2005;  Rosas  et  al.,  2003;  Zheng,  2006),  in  afternoon  school  programs 
in  youth  facilities  outside  of  school  (Squire  et  ah,  2005),  or  in  learners’  homes  (McFarlane  et  ah,  2002).  Different 
settings  have  different  dynamics  and  social  cultural  rules.  What  strategies  are  needed  to  implement  educational 
computer  games  in  varying  settings?  How  should  teachers  and  administrators  be  involved  and  supported?  Are  some 
games  more  appropriate  for  one  setting  than  another? 

Facilitator  Knowledge,  Attitude,  Roles 

Previous  research  on  educational  games  indicates  that  the  facilitator  is  a key  factor  impacting  the  outcome 
of  educational  use  of  games  in  the  school  setting  (Bredemeier  & Greenblat,  1981;  Egenfeldt-nielsen,  2005). 
Facilitators’  attitude  toward  games  and  their  knowledge  about  games  as  well  as  the  subject  matter  makes  a 
significant  difference  in  students’  learning.  Egenfeldt-nielsen  (2005)  found  that  the  facilitator  who  has  more 
experience  with  computer  games  and  who  has  a view  of  history  as  more  than  historical  facts  tend  to  be  more 
successful  in  facilitating  history  learning  from  a computer  game.  In  a study  described  by  Squire  and  colleagues 
(2005),  facilitators  were  expert  players  of  Civilization  3. 

Existing  research  shows  that  facilitators  play  various  important  roles  in  learning  environments  enhanced  by 
computer  games.  For  example,  in  an  after-school  program  in  which  student  participants  played  a strategy  game  Civ 
3 to  learn  history  (Squire  et  al.,  2005),  facilitators  played  the  role  of  teachers  and  cheerleaders  at  the  beginning  of  the 
program,  and  they  became  more  like  mentors  or  expert  players  as  students  gained  more  competence.  The  facilitator 
sometimes  played  the  game  along  with  the  students.  Egenfeldt-nielsen  (2005)  also  found  that  teachers  play  many 
roles.  They  had  to  solve  computer  problems  and  helped  students  playing  the  game.  At  the  same  time,  they  had  a 
critical  role  of  helping  students  appreciate  and  explore  historical  information  and  dynamics  as  well  as  make  the 
connection  between  game  playing  and  scientific  concepts.  Facilitators  may  need  help  to  acquire  the  knowledge  and 
skills  required  to  perform  their  roles.  What  support  do  facilitators  need?  How  should  the  support  be  provided  to 
facilitators?  For  example,  should  it  be  provided  through  training,  just-in-time  tutorial  and  resources,  or  a 
combination  of  different  strategies? 
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Addressing  Learner  Differences 


Computer  games  are  arguably  the  most  popular  media  among  children.  However,  there  are  many  individual 
differences  when  it  comes  to  educational  computer  games.  Squire  and  colleagues  (2005)  found  that  middle  class 
boys  reported  dreams  to  become  game  designers,  whereas  children  with  lower  social-economic  backgrounds  were 
challenged  by  games  and  did  not  have  game  design  aspirations.  Similarly,  in  Denmark,  Egenfeldt-nielsen  (2005) 
found  that  some  students  had  little  experience  with  computer  games.  They  gave  up  on  the  educational  computer 
games  played  during  the  study.  Only  a few  students  succeeded  in  this  class.  Those  that  succeeded  in  the  class 
generally  had  more  experience  with  computer  games,  liked  history,  and  had  extra  energy  to  engage  in  learning. 
Learners  in  educational  computer  games  may  have  different  levels  of  experience,  skill,  and  self-efficacy  with  games. 
How  can  educational  computer  games  be  designed  and  implemented  to  meet  learners’  varying  needs?  Lai,  Ma,  and 
Williams  (October  2006)  generated  some  design  guidelines  to  improve  learners’  self-efficacy  in  educational 
computer  games.  How  effective  are  these  strategies?  More  research  is  needed  in  this  area. 
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Methodology 


Design-based  research  may  provide  a framework  for  an  effective  methodology  in  researching  educational 
games.  Design-based  research  emphasizes  the  design  of  innovative  learning  environments  based  on  theory  and 
empirical  research  through  an  iterative  process  of  design,  implementation,  analysis,  and  redesign  (Design-Based 
Research  Collective,  2003).  The  purpose  of  this  iterative  process  is  not  only  to  enhance  the  particular  intervention 
being  investigated  but  also  to  develop  theories  to  account  for  the  impact  of  the  intervention  and  to  create  models  to 
inform  the  design  of  other  innovations.  Design-based  research  requires  collaboration  among  practitioners, 
researchers,  and  technologists  (Reeves,  Herrington,  & Oliver,  2004).  This  emerging  methodology  has  been  used  to 
inform  the  design  and  investigate  the  impact  of  such  innovative  educational  programs  as  The  Adventures  of  Jasper 
Woodbu ry  (Cognition  and  Technology  Group  at  Vanderbilt  [CTGV])  and  GenScope  (Hickey,  Kindfield,  Horwitz,  & 
Christie,  2003). 

To  address  the  research  issues  presented  in  this  paper,  both  qualitative  and  quantitative  methods  may  be 
needed  to  guide  data  gathering  and  analysis.  These  methods  are  appropriate  for  different  stages  of  design-based 
research  (Bannan-Ritland,  2003;  Design-Based  Research  Collective,  2003;  Kelly,  2004;  Shavelson,  Phillips,  Towne, 
& Feuer,  2003).  Qualitative  methods  are  especially  helpful  during  the  exploration  phase  of  a design-based  research 
project  (Kelly,  2004;  Shavelson  et  ah,  2003)  when  models  (Sloane  & Gorard,  2003),  conjectures  (Sandoval,  2004), 
or  hypothesis  (Kelly,  2004)  are  formulated  in  the  context  of  real  world  problems  and  interventions.  Quantitative 
methods  are  helpful  later  when  knowledge  that  emerged  from  qualitative  explorations  is  validated. 
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Innovative  Multiple  Outcome  Design: 

Using  Narrative  Simulation  to  Train  P-12  Teacher  Mentors 
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Abstract 

Given  the  increase  in  P-12  education  reform  efforts  over  the  last  decade,  mentoring  new  teacher  interns  can  be  a 
daunting  responsibility.  Experienced  teachers,  in  training  to  be  mentors,  often  find  themselves  inundated  with  large 
amounts  of  didactic  information  as  they  prepare  for  guiding  new  interns.  In  an  effort  to  develop  a more  effective 
means  of  training  mentors,  the  Educational  Professional  Standards  Board  (EPSB)  in  one  state  enlisted  the  help  of 
instructional  designers  to  develop  an  online  course  that  would  effectively  prepare  new  mentors  and  revitalize 
experienced  mentors.  The  focal  point  for  the  online  course  is  a narrative  simulation  embedded  with  standards-based 
information,  designed  to  promote  reflection  and  offer  opportunity  for  practice  making  difficult  decisions.  As  a 
fulcrum  for  the  online  course,  the  narrative  simulation  employs  an  innovative  multiple  outcome  design  to  frame 
dialogic  interaction  between  the  mentor  and  the  instructional  content.  In  this  paper,  we  discuss  theoretical  and 
practical  considerations  for  using  multiple  outcome  decision  points  to  support  dialogic  interaction  within  a narrative 
simulation. 


Introduction 

As  demands  for  accountability  increase,  P-12  education  is  flooded  with  high-stakes  tests,  regulations, 
specified  curriculum,  instructional  guidelines,  and  performance  standards  for  teachers.  There  is  the  charge  of  No 
Child  Left  Behind  to  assure  that  all  students  perform  at  the  same  minimum  academic  standards,  regardless  of  ability, 
preparation,  support  outside  of  school,  or  opportunity.  There  are  national  standards  for  public  education,  state 
mandates  for  student  achievement,  and  local  demands  that  hold  school  systems,  schools,  administrators,  and 
teachers  accountable  for  reducing  variance  in  student  performance.  Whatever  the  intent  of  such  demands,  a myopic 
focus  on  standardizing  student  performance  can  create  the  impression  that  there  are  best  practices  to  apply  across 
contexts,  events,  teachers,  and  students.  In  fact,  educational  leaders  at  local,  state,  and  national  levels  produce  an 
enormous  amount  of  textual  information  (online  and  in  paper  form)  presumably  in  an  effort  to  establish  best 
practices  for  P-12  classrooms.  These  large  amounts  of  didactic  information  often  overload  new  teacher  interns  (first 
year  teachers),  as  well  as  experienced  teachers  serving  as  their  mentors,  leaving  little  time  for  the  kind  of  training 
that  may  better  address  teaching  and  learning  in  widely  diverse  real  world  contexts. 

New  teachers  and  their  mentors  are  left  to  navigate  this  maze  of  information,  prioritize  it  and  inteipret  it  in 
their  day-to-day  instructional  practice.  To  characterize  the  situation  of  these  teachers  and  their  mentors,  in 
instructional  design  terms,  it  is  as  though  design  and  development  of  mentor  training  begins  and  ends  with 
producing  and  refining  content,  with  little  attention  to  how  such  content  can  be  delivered  most  effectively. 
Recognizing  the  need  for  better  delivery  mechanisms  in  training  teacher  mentors,  a state  Educational  Professional 
Standards  Board  (EPSB)  contracted  with  university-based  instructional  designers  to  assist  in  developing  an 
innovative  and  effective  online  mentor  training  course.  The  goal  was  to  design  an  online  course  that  would  both 
effectively  prepare  new  mentors  and  revitalize  experienced  mentors.  The  focal  point  for  the  online  course  is  a 
multiple  outcome  narrative  simulation  embedded  with  standards-based  information  that  is  designed  to  offer 
opportunities  for  practice  making  difficult  decisions. 

Design  Challenges 

Given  this  task,  we  began  by  examining  the  form  and  content  of  previous  mentor  trainings,  reviewing  and 
prioritizing  content,  and  initiating  conversations  with  experienced  mentors.  We  explored  state  teacher  standards, 
new  teacher  expectations,  mentor  training  materials  and  agendas  for  past  training  sessions.  While  the  content  of  all 
was  easy  to  understand,  the  amount  of  information  and  the  redundancy  with  which  it  was  organized  presented 
several  difficult  design  challenges.  For  example,  there  was  relatively  little  content  that  addressed  implementation, 
which  left  a gap  between  expectations  and  actual  learning  activities  that  would  prepare  mentors  to  guide  interns.  In 
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fact,  during  our  initial  conversations  with  experienced  mentors,  most  indicated  a high  level  of  frustration  with 
previous  trainings,  because  of  the  constantly  revised  and  unwieldy  amount  of  standards-based  content.  Moreover, 
the  training  lacked  opportunities  to  prepare  for  the  more  difficult  unanticipated  situations  that  occur  in  mentoring 
new  teachers. 

Our  major  instructional  challenges  included:  (1)  presenting  large  amounts  of  prescribed  standards-based 
content,  (2)  engaging  learners  who  were  all  experienced  teachers,  some  new  to  mentoring  and  others  who  had  been 
mentors  in  the  past,  (3)  providing  opportunity  for  practice  solving  unanticipated  problems,  and  (4)  facilitating 
dialogic  interaction  both  among  learners  and  with  the  course  content.  Additionally,  the  course  needed  to  be  suitable 
for  long  term  use  with  only  minor  revisions.  In  order  to  make  the  course  engaging  for  both  experienced  and  new 
mentors,  we  situated  the  content  (provided  to  us  by  the  EPSB)  in  an  online  interactive  environment  that  revolves 
around  a narrative  simulation  based  on  an  actual  case  of  mentoring  {Staying  Past  Dark).  We  distilled  the  storyline, 
events,  characters,  and  critical  dilemmas  from  a case  study  of  a new  teacher  and  a mentor  who  struggled  with  a 
variety  of  difficult  challenges  (Jones,  2003).  After  condensing  the  case,  we  rendered  it  as  a narrative  simulation  with 
a range  of  strategically  placed  information  on  new  teacher  standards,  guidelines  and  expectations.  Staying  Past  Dark 
sets  the  stage  for  several  subsequent  modules  in  the  interactive  course,  which  combines  a variety  of  problem  solving 
activities,  research  to  practice  information,  and  self-assessments,  as  summarized  below. 


1.  The  Community  of  Practice  framework  (Wenger  and  Snyder,  2000). 

2.  Staying  Past  Dark,  a narrative  simulation  that  is  based  on  actual  events  experienced  by 
mentors  faced  with  many  mentoring  issues  and  decisions. 

3.  Information  and  exercises  that  focus  your  exploration  of  the  practice  of  mentoring  as 
critical  inquiry  and  collaborative  problem  solving.  Specifically  this  means  that  we  will 
discuss  activities  that  explore  dialoging,  conferencing,  questioning,  and  critical  thinking 
as  key  elements  in  mentoring.  Collaborative  responsibility  of  all  participants  in  a 
mentoring  experience  is  emphasized. 

4.  Assessments  that  are  professionally  and  personally  challenging,  requiring  you  integrate 
your  insight,  experience,  and  knowledge  of  course  content. 

5.  The  Work  Sample  approach  (replacing  the  portfolio  documentation  used  previously  in 
mentor  training)  that  will  be  used  to  evaluate  interns  during  year  two  of  the  pilot  project. 

Making  informed  decisions  in  novel,  varied,  or  unexpected  situations  requires  a different  kind  of  mental 
activity  than  that  involved  in  knowing  or  remembering  prescribed  standards  and  practices.  The  rationale  for 
combining  standards-based  didactic  information  with  problem-solving  practice  was  based  on  the  fact  that  mentors 
not  only  have  to  refer  to  the  current  expectations  for  new  teachers,  they  must  also  assist  interns  in  applying 
standards-based  requirements  to  a range  of  circumstances.  Such  enactment  involves  productive  thinking  that  results 
in  the  ability  to  transfer  knowledge  rather  than  reproductive  thinking  that  may  promote  problem  solving  in  similar 
situations  but  does  not  transfer  to  novel  instances  or  events.  In  fact,  a long-term  challenge  in  teacher  preparation  is 
that  many  day-to-day  decisions  in  the  classroom  cannot  be  standardized  because  they  are  contingent  on  student 
responses  and  particular  objectives  at  any  given  moment  (Hammerness,  Darling-Hammond,  and  Shulman,  2005). 
While  meeting  standards  and  new  teacher  expectations  is  essential  for  interns,  they  are  faced  with  unique  problems 
on  a daily  basis.  Mentors  must  be  prepared  to  help  interns  make  difficult  decisions  when  there  is  no  prescribed 
solution. 


Conceptualizing  Narrative  Simulation  Design  for  Dialogic  Learning 
Characteristics  of  Effective  Mentors 

Recent  research  on  mentoring  has  articulated  various  characteristics  of  effective  mentors  (Feiman-Nemser, 
Schwille,  Carver,  and  Yusko,  1999).  These  characteristics  include  the  ability  to:  (1)  reflect  about  one's  own  practice, 
(2)  articulate  goals,  strengths  and  growth  areas,  (3)  establish  a strong  foundation  for  working  together,  (4)  use  a 
variety  of  teacher  assessment  instillments,  (5)  analyze  and  prioritize  assessment  data  objectively,  and  (6)  listen 
carefully  and  question  judiciously  during  conferences.  In  order  to  meet  such  tall  orders,  mentors  must  understand 
and  handle  a complex  of  situations  that  require  reflective  reasoning  and  professional  judgment.  For  example,  they 
need  to  know  how  to  deal  with  their  own  first  impressions  of  a new  colleague  they  are  asked  to  mentor,  exploring 
their  own  biases,  as  well  as  those  of  the  new  teacher  they  mentor.  Mentors  also  must  understand  the  phases  of  new 


294 


teacher  experience,  how  to  conduct  strategic  observations,  how  to  articulate  areas  for  professional  growth,  and  how 
to  intervene  when  problems  occur. 

Using  Narrative  to  Understand  Experience 

In  order  to  create  an  interactive  environment  that  would  accommodate  large  amounts  of  didactic 
(standards-based)  information  and  promote  reflective  thinking,  we  choose  a narrative  framework  for  the  online 
course.  We  used  narrative  because  of  its  potential  to  be  engaging,  encourage  reflective  thinking,  and  situate 
information  in  familiar  contexts  (Brown,  Collins,  and  Duguid,  1989;  Cognition  and  Technology  Group  at 
Vanderbilt,  1990,  1993;  Lave  and  Wenger,  1991;  Polkinghorne,  1988;  Young,  1993).  According  to  Bruner  (1990) 
narrative  promotes  a different  kind  of  thinking  than  didactic  information.  While  presenting  information  as  facts  can 
lead  to  logical  or  paradigmatic  modes  of  thought,  it  leaves  little  room  for  interpretation  or  imagination.  Narrative, 
however,  can  represent  human  behavior  and  events  as  lived  experience  in  a manner  that  offers  opportunity  for 
interpretation  and  imagination,  prompting  learners  to  infer  meanings  that  are  not  explicit.  In  fact,  many  insist  that 
human  beings  think  in  narrative  form  and  that  some  of  our  most  memorable  experiences  are  held  in  mind  as  stories 
(Vitz,  1990;  Sarbin,  1986;  Spence,  1982;  Fisher,  1995).  Epistemologically,  the  use  of  narrative  in  instruction  relies 
on  a constructivist  perspective  or  the  view  that  knowledge  is  constructed  rather  than  transmitted.  From  this 
perspective,  learners  play  an  active  role  in  their  own  learning  (Duffy  and  Cunningham,  1996).  Finally,  we  view 
learning  as  a social  process  that  is  enhanced  by  interaction  and  dialogue  (Jonassen,  1999;  Hatano  and  Inagaki,  1993; 
Vygotsky,  1978). 

Narrative  and  Problem  Based  Learning 

Narrative  has  been  used  extensively  in  problem-based  instruction  and  has  been  found  to  improve 
comprehension  and  transfer  (Conle,  2003;  Egan,  1988;  Laurillard,  1998).  One  important  use  of  narrative  has  been  in 
goal-based  scenarios  (Schank,  Fano,  Bell,  and  Jona,  1993).  Goal-based  scenarios  (GBS)  are  designed  to  develop 
skill  and  content  knowledge  by  simulating  events  in  situated  contexts  (Schank,  Berman,  and  Macpherson,  1999). 
Unlike  narrative  simulation,  GBS  require  users  to  work  toward  a prescribed  goal  as  characters  with  predetermined 
roles.  Although  GBS  employs  realistic  narrative,  it  can  be  fictitious  and  stimulate  game-playing  strategic  thinking 
rather  than  the  kinds  of  critical,  imaginative,  and  reflective  decision-making  required  in  narrative  simulation 
(McCrary,  2002). 

As  an  instructional  intervention,  narrative  simulation  has  a long  history  of  effectiveness  in  accident 
prevention,  farming  and  mining  safety,  and  solutions  to  work  place  problems,  such  as  sexual  harassment  (Cole, 

1997;  2002).  It  has  been  particularly  effective  in  personalizing  new  information,  changing  practices,  and  shifting 
attitudes  (McCrary,  2002:  McCrary  and  Mazur,  1999).  As  simulation  stories  develop,  dilemmas  evolve,  which 
engage  learners  in  real  world  problems,  decision-making,  and  solutions.  Dilemmas  drive  this  type  of  learning  and 
learners  literally  become  characters  in  an  unfolding  story.  It  involves  (1)  learning  that  is  grounded  in  everyday 
activities,  (2)  situational  knowledge  that  transfers  to  similar  circumstances,  (3)  learning  that  results  from  a social 
process  involving  certain  ways  of  thinking,  perceiving,  problem-solving,  and  interacting,  and  (4)  learning  that  exists 
in  complex  social  environments  made  up  of  actors,  actions,  and  circumstances. 

Traditionally,  narrative  simulations  require  learners  to  read  segments  of  a story,  followed  by  a question, 
and  select  a course  of  action  by  choosing  from  a list  that  includes  correct  and  incorrect  answers.  More  recently, 
design  approaches  for  narrative  simulation  have  been  effective  in  providing  learners  with  practice  making  difficult 
decisions  even  when  there  is  no  single  correct  course  of  action.  For  example,  studies  on  the  use  of  Jeffs  Story,  a 
narrative  simulation  on  social  justice,  have  demonstrated  its  effectiveness  in  personalizing  difficult  social  problems 
that  have  otherwise  been  considered  remote  or  irrelevant  by  the  learner  (McCrary,  2002).  Given  this  evolution  of 
instructional  narrative,  we  went  a step  further  in  developing  a teacher  mentoring  simulation  with  divergent 
outcomes.  Unlike  traditional  narrative  simulations  with  a single  outcome,  Staying  Past  Dark  departs  from  the  main 
storyline  at  a most  critical  point  and  allows  users  to  choose  one  of  three  narrative  paths,  each  resulting  in  a distinct 
outcome  based  on  the  users'  choice  of  action.  The  learner  is  prompted  to  choose  an  ending  based  on  what  s/he 
anticipates  the  outcome  of  the  problem  situation  will  be.  The  simulation  thus  becomes  a fulcrum  for  dialogic 
interaction  between  the  mentor  and  the  instructional  material.  Reflection  on  the  simulated  problem  is  personalized 
and  operationalized  as  the  mentor  projects  an  outcome.  In  the  sections  that  follow  we  focus  on  the  theoretical  and 
practical  considerations  for  using  multiple  outcome  decision  points  to  promote  dialogic  interaction  within  a narrative 
simulation. 
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Dialogic  Learning  and  Professional  Reasoning 

Dialogic  learning  has  been  extant  since  ancient  times  (van  der  Linden,  2005).  The  Socratic  dialogue 
technique  is  included  in  most  teacher  preparation  curriculum  and  instruction  or  pedagogical  methods  course.  Recent 
research  on  dialogic  learning  has  elaborated  the  complexities,  benefits,  and  challenges  of  using  dialogic  approaches 
in  educational  settings  that  have  institutionalized  positivistic  pedagogies  to  effect  didactic  instruction  (Ruf  and  Badr 
Goetz,  2002).  The  use  of  dialogue  is  often  couched  as  question  and  answer  interaction.  A close  examination  of  the 
etymology  of  the  word  dialogue,  however,  reveals  the  complex  and  layered  dimensions  possible  in  dialogic  learning. 
Dia,  from  the  Greek  meaning,  two  and  logic  meaning,  reasoning.  It  is  this  reasoning  that  was  a crucial  element  in 
designing  the  narrative  simulation  for  teacher  mentors.  Mentors  and  interns  are  required  to  go  beyond  decision- 
making and  reflect  deeply  to  examine  and  analyze  how  those  decisions  embody  their  theories-in-action  and 
professional  reasoning  in  the  classroom. 

Design:  Multiple  Outcomes  for  a Narrative  Simulations 

Based  on  composite  information  from  experienced  mentors  during  our  developmental  research,  we  found 
that  mentors,  as  experienced  teachers,  often  anticipated  outcomes  or  wondered  about  the  success  of  an  intern  when 
problems  occured.  Mentoring  training  often  focused  on  techniques  such  as  conducting  a post-observational 
conference,  but  did  not  provide  opportunitites  to  puzzle  about  or  work  through  those  nagging  internal  questions? 
Would  the  new  teacher  make  it?  Would  the  new  teacher  truly  change  her  practice  to  meet  the  challenges  of  a 
difficult  class?  Would  she  leave  teaching  altogether?  Or,  would  she  just  bide  her  time,  thinking  the  criticisms  would 
blow  over,  and  hope  for  a better  group  of  students  next  year?  These  are  the  dilemmas  and  experiences  we  sought  to 
capture  and  address  by  using  a multiple  outcome  design  within  a narrative  simulation.  By  selecting  an  outcome  to 
the  year-long  mentoring  situation  portrayed  in  the  simulation,  the  mentor  exposes  his  or  her  cognitive  and  affective 
responses  to  the  real-world  situation  posed  in  the  story.  Critical  aspects  of  the  mentoring  relationship  can  be 
examined  in  light  of  the  mentor’s  choice. 

Staying  Past  Dark:  A Case  for  Training  Teacher  Mentors 

Staying  Past  Dark  focuses  on  a case  in  which  the  first  year  teacher  experienced  a range  of  problems  and  the 
mentor  was  faced  with  difficult  decisions.  In  order  to  create  a qualitatively  different  instructional  environment  aimed 
at  encouraging  critical  forethought  (simulation)  instead  of  informed  hindsight  (case  study),  we  condensed  the  actual 
story  of  mentoring,  determined  critical  decision  points,  added  corresponding  question  and  answer  choices,  and 
employed  first-person,  present-tense  language  (see  figure  1). 

Figure  1.  Sample  story  segment  of  narrative  simulation. 

Staying  Past  Dark 
The  Beginning 

I let  central  office  know  I would  be  willing  to  serve  as  a mentor  for  a new  teacher.  When  they  assigned  Kristin  to 
me,  I told  them  I looked  forward  to  the  experience. 

I met  with  Kristen  right  away  and  we  talked  about  how  this  internship  would  go.  She  was  already  nervous  about  her 
first  year  of  teaching  because  she  heard  her  fourth  graders  were  out-of-control.  This  group  of  kids  had  been  terrors 
in  the  third  grade  and  no  one  had  been  able  get  them  under  control. 

I didn't  know  how  to  respond  to  her  concerns  because  I couldn't  tell  if  they  really  assigned  a new  teacher  an 
impossible  class  or  if  she  was  just  having  first  year  jitters. 

Please  read  the  following  question  and  select  the  response  that  best  represents  what  you  would  do  as  Kristen's 
mentor. 

Question  A:  As  Parker  Olsen,  the  mentor,  how  would  you  have  responded  to  Kristen's  initial  concerns? 

1 . 1 would  meet  with  the  principal  right  away  and  discuss  Kristen's  concerns,  assigning  a first  year 
teacher  an  impossible  class,  and  the  fact  that  she  might  need  extra  help  managing  a class  that  no 
one  else  had  been  able  to  control. 
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2. 1 would  talk  with  Kristen  more  about  her  concerns  and  question  her  to  try  to  find  out  more 
about  the  situation. 

3. 1 would  just  listen,  say  nothing,  and  wait  to  see  what  happened  when  I observed  her  teaching. 

4. 1 would  speak  with  the  third  grade  teacher  who  had  these  kids  last  year  and  try  to  find  out  if 
Kristen's  fears  were  on  target. 

STANDARD  II:  The  Teacher  Creates  and  Maintains  a Learning  Climate  for  Students 

A.  The  teacher  communicates  high  expectations  for  all  students. 

B.  The  teacher  supports  student  diversity  and  addresses  individual  needs. 

C.  The  teacher  uses  positive  classroom  management  techniques  that  foster  self-control  and  self- 
discipline  to  create  and  sustain  a climate  that  motivates  students  to  learn. 

D.  The  teacher  facilitates  mutual  respect  among  class  members  through  cooperative  and 
independent  learning  activities. 

E.  The  teacher  employs  creative  and  flexible  use  of  instructional  time  and  materials. 

F.  The  teacher  supports  instruction  through  the  creative,  flexible,  and  safe  use  of  physical  space. 


Incorporating  the  Multiple  Outcomes  Design  Component 

After  working  through  various  decision  points  within  the  narrative  simulation  each  learner  departs  from  the 
main  storyline  based  on  a selected  response  to  a dilemma-based  question  midway  through  the  exercise.  From  that 
point,  users  follow  one  of  three  possible  narrative  paths,  each  resulting  in  a different  outcome  to  the  story.  We 
derived  the  two  additional  real  world  outcomes  from  stories  told  by  experienced  educators  during  our  developmental 
research  sessions.  Figure  2 shows  the  divergent  outcome  segment  of  the  narrative  simulation,  followed  by 
introductions  for  each  possible  path. 

Figure  2.  Divergent  outcome  segment  of  narrative  simulation. 

Now  it's  your  turn! 

How  do  you  think  this  story  will  end? 

Will  Parker's  plan  work  and  will  Kristen  remain  in  teaching? 

Click  on  one  of  the  following  story  endings  and  follow  that  scenario  to  the  END. 


Who’d  Have 

A Sad 

Loosing  a 

Thought 

Realization 

New  Teacher 

Wednesday  January 
Who  'cl  Have  Thought 

I developed  four  components  of  a work  sample,  1)  contextual  factors,  2)  unit  overview  with  learner  objectives,  3) 
lesson  plan,  and  3)  assessment  plan.  1 used  materials  from  my  own  teaching  files  and  updated  the  formats  to  reflect 
current  KTIP  criteria  as  a way  to  provide  Kristen  with  concrete  examples  for  her  to  use  in  making  her  own  work 
sample. 

I went  over  the  School  Performance  Report  for  Greenbrier  Elementary  and  suggested  she  also  use  that  report  to 
develop  the  contextual  factors  component.  When  she  saw  the  numerical  and  graphic  data  about  achievement  gaps, 
content  comparisons,  and  demographics,  the  light  bulb  went  on.  She  could  see  how  this  report  could  be  used  to 
improve  her  instructional  planning.  She  said  she  thought  it  would  be  fun  to  really  explore  contextual  factors  for  her 
group  of  students.  She  seemed  to  have  a much  better  attitude  after  the  Christmas  break! 
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Wednesday  January  9 
A Sad  Realization 

I developed  four  components  of  a work  sample,  1)  contextual  factors,  2)  unit  overview  with  learner  objectives,  3) 
lesson  plan,  and  3)  assessment  plan.  I used  materials  from  my  own  teaching  files  and  updated  the  formats  to  reflect 
current  KTIP  criteria  as  a way  to  provide  Kristen  with  concrete  examples  for  her  to  use  in  making  her  own  work 
sample. 

When  we  met,  I gave  her  my  materials,  and  asked  her  to  begin  doing  her  own  work  sample  by  first  completing  the 
contextual  factors'  component  and  having  it  ready  for  our  next  meeting  in  two  weeks.  I went  over  the  School 
Progress  Report  with  her  and  she  said  she  thought  it  would  be  fun  to  really  explore  contextual  factors  for  her  group 
of  students.  She  seemed  to  have  a much  better  attitude  after  the  Christmas  break! 


Wednesday  January  9 
Loosing  a New  Teacher 

I developed  four  components  of  a work  sample,  1)  contextual  factors,  2)  unit  overview  with  learner  objectives,  3) 
lesson  plan,  and  3)  assessment  plan.  I used  materials  from  my  own  teaching  files  and  updated  the  formats  to  reflect 
current  KTIP  criteria  as  a way  to  provide  Kristen  with  concrete  examples  for  her  to  use  in  making  her  own  work 
sample.  When  we  met,  I gave  her  my  materials,  and  asked  her  to  begin  doing  her  own  work  sample  by  first 
completing  the  contextual  factors'  component  and  having  it  ready  for  our  next  meeting  in  two  weeks.  I went  over  my 
examples  with  her  and  she  said  she  thought  it  would  be  fun  to  really  explore  contextual  factors  for  her  group  of 
students.  She  seemed  to  have  a much  better  attitude  after  the  Christmas  break! 


As  we  developed  Staying  Past  Dark  certain  criteria  guided  us  through  several  formative  evaluations.  Those 
criteria  included  fidelity  to  the  actual  story  events  to  retain  a sense  of  veracity  for  experienced  educators.  The 
narrative  simulation,  even  when  diverging  into  three  possible  outcomes,  had  to  remain  believable  to  these 
experienced  professionals  to  maintain  their  interest  and  motivation  to  work  through  the  entire  exercise  and  take 
advantage  of  the  embedded  didactic  information. 

Design  Criteria  for  Using  Divergent  Multiple  Outcomes  in  a Narrative  Simulation 

While  real  world  stories  must  maintain  fidelity  when  transformed  to  narrative  simulations,  it  is  also 
important  that  stories  be  condensed  by  removing  extraneous  information.  The  aesthetic  power  of  narrative,  however, 
can  be  diminished  if  a balance  is  not  carefully  preserved  between  the  inclusion  of  descriptive  language  and  concern 
for  instructional  efficacy.  The  primary  power  of  narrative  lies  in  both  its  believability  and  the  ways  it  can  inspire 
readers  to  imagine,  to  see,  and  to  feel  the  story.  Designing  narrative  simulation  for  instruction  requires  that  the  story 
be  segmented  into  brief  accounts,  each  segment  including  a dilemma  that  can  be  followed  by  a question  and  possible 
response  choices.  In  this  case,  we  took  the  main  storyline  from  a case  study  on  a new  teacher  intern.  We  also  had  to 
assure  that  affective  qualities,  such  as  descriptions  of  characters,  setting,  and  emotions,  inspired  compassion  and 
empathy.  With  these  criteria  for  story-based  simulation  in  mind,  we  needed  to  also  elaborate  criteria  for  adding  the 
divergent  multiple  outcomes  component.  The  iterative  process  of  developing  Staying  Past  Dark  resulted  in  the 
formulation  of  the  initial  design  criteria  for  the  multiple  outcomes  approach  shown  in  Table  1 below. 

Table  1.  Design  Criteria  for  Narrative  Simulations  for  Dialogic  Learning:  An  Initial  Formulation 


Design  Criteria  Design  Process/Product  Characteristics 

Simulation  Fidelity  Case -based  and  rate  based  data  used  to  develop  story  line 

Simulation  Veracity  Narrative  based  on  actual,  real-world  situations,  data  from  formative 

evaluation  and  user-centered  design  required 

Definition  of  Narrative  (rather  • Affective  outcomes  for  users  defined,  based  on  characteristics  of 
than  paradigmatic)  Learning  professional  mastery  (e.g.  for  teacher  mentors,  recognition  of  bias) 

Outcomes 
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Narrative  Structure  for  Content 
Engagement  with  Narrative 


Opportunity  for  Dialogic 
Interaction:  Internal 


Opportunity  for  Dialogic 
Interaction:  External 


• Cognitive  outcomes  for  users  defined,  based  on  required  cognitive 
knowledge  and  skill  for  user  mastery  (e.g.  for  teacher  mentors,  ability  to 
revise  instruction  based  on  learning  outcomes) 

• Instructions  to  read  the  story  in  sequence  and  not  skip  ahead 

• Story  events  occur  over  time 

• User  assumes  the  role  of  a character  in  an  unfolding  story. 

• Chunking  story  line  to  pose  incremental  elements  of  the  dilemma 

• Plausible  decision  points  for  user-choices. 

• Feedback  based  on  user-choices 

• Opportunities  to  explore  other  choice  responses 

• Definition  of  multiple,  plausible  outcomes  from  user-centered  design 
processes 

• Inclusion  of  prompts  to  spur  reflection  and  analysis  of  the  dilemma  and 
resolutions 

• Definition  of  multiple,  plausible  outcomes  from  user-centered  design 
processes 

• Inclusion  of  prompts  to  spur  reflection  and  analysis  of  the  dilemma  and 
resolutions 

• Inclusion  of  collaboration  and  communication  tools  to  encourage  dialogue 
among  users  of  the  narrative  simulation 

• Archiving  of  dialogues  (e.g.  in  a threaded  discussion  group)  for  ongoing 

dialogue,  reflection  and  interaction.  Archiving  also  provides  opportunities 
for  those  not  involved  in  original  dialogue  to  participate  vicariously. 


Discussion 

Staying  Past  Dark  delivers  an  otherwise  confusing  maze  of  teacher  standards,  student  achievement 
expectations,  and  professional  ethics  in  a more  personally  relevant  manner  as  embedded  text  annotations  throughout 
the  unfolding  story.  The  inclusion  of  Staying  Past  Dark  in  this  online  mentoring  course  has  been  successful  in 
transforming  the  traditional  EPSB  instruction  from  a more  static  and  often  redundant  experience  to  one  that  engages 
mentors  in  an  active  and  reflective  discourse  on  teaching  and  learning.  It  is  more  congruent  with  the  actual  practice 
of  applying  standards  to  mentoring  intern  teachers  in  varied  contexts.  Were  it  true  that  human  development  could  be 
standardized,  then  learning  to  become  effective  teachers  might  lend  itself  to  more  traditional  didactic  instruction.  In 
fact,  learning  to  mentor  new  teachers  would  be  a relatively  easy  task,  requiring  concrete  knowledge  of  correct 
practices  to  meet  predetermined  goals.  When  charged  with  training  experienced  educators  to  effectively  mentor 
novice  teacher  interns,  however,  it  becomes  clear  that  organizing  and  distributing  state  teacher  standards  and  new 
teacher  expectations  does  little  to  help  mentors  guide  interns  in  the  day-to-day  enactment  of  those  standards.  More 
importantly,  mentors  must  assist  their  new  teacher  colleagues  in  becoming  confident  problem  solvers,  who  can 
wisely  reason  through  the  complexities  inherent  in  the  myriad  classroom  activities  in  which  engage  each  day. 
Narrative  simulations  with  divergent  multiple  outcomes  provide  one  opportunity  for  the  mentors  to  explore  these 
complexities  and  reflect  on  their  own  decision-making  by  providing  opportunities  for  both  internal  and  external 
dialogic  reasoning.  These  conversations,  both  internal  and  external,  are  the  substance  of  professional  judgment  that 
rarely  occurs  through  traditional,  didactic,  non-dialogic  instruction. 
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Deeper  Learning  Through  a Contribution  Model  for  Learning  Object  Design 

Patricia  McGee 

Abstract.  The  proliferation  of  learning  object  repositories  while  useful  has  not 
necessarily  resulted  in  exemplary  instructional  design.  This  paper  puts  forth  a 
learning  object  pedagogical  design  that  is  based  on  the  contribution  of  the 
learner.  Through  thoughtful,  engaged,  active,  and  purposeful  learning,  the 
learner  constructs  understanding  and  creates  an  object  that  becomes  part  of  the 
object’s  learning  sequence.  In  this  way  the  learner  has  ownership  of  what  they 
know  in  a manner  that  contributes  to  the  learning  of  others. 

Moving  curricula  from  the  classroom  to  digital  formats  requires  management,  oversight,  and  learning 
designs  for  access  and  dissemination  through  knowledge  systems.  The  focus  on  knowledge  management  in  higher 
education  can  be  seen  in  the  trend  toward  learning  objects.  Although  defined  in  a number  of  ways,  depending  on  the 
context  and  culture  from  which  they  emerge,  the  author  defines  a learning  object  as  any  digital  asset  that  is  intended 
to  be  used  to  support  learning,  can  be  modified,  and  can  be  re-used  in  different  contexts  for  different  purposes. 
Learning  objects  should  be  re -useable  and  re-purposeable  over  time  and  location  and  interoperable  across  systems 
and  software  (see  Robson,  2005).  Unlike  other  technology  products  that  support  learning,  such  as  learning/course 
management  systems  (L/CMS),  learning  objects  have  been  conceptualized  in  an  era  of  standardization  and 
interoperability.  Soon  after  the  adoption  of  the  term  learning  object,  designers,  developers,  and  managers  stressed 
the  reusable  and  interoperable  attributes  of  a learning  object  but  as  standardization  and  specifications  begin  to  be 
adopted,  these  attributes  are  less  of  a focus.  They  speak  to  the  underlying  assumptions  about  the  use  of  technology  to 
support  learning  - how  it  works  has  generally  come  before  how  it  is  designed  and  used  for  learning.  The 
contributions  of  the  2002  work  of  Wiley  and  associates  has  not  led  us  to  universal  strategies  that  are  specific  to 
objects,  but  rather  tactics  that  can  be  used  with  either  multimedia  or  learning  systems  in  general.  This  paper  argues 
for  a learning  design  that  is  contributory  and  then  describes  a formative  project  that  embraces  principles  of  a 
contribution  pedagogy  while  supporting  deeper  learning. 

Although  current  discourse  and  literature  tends  to  deeper  learning  as  a desired  outcome  (see  Carmean  & 
Haefner,  2002;  Van  Weigel,  2003)  what  actually  happens  when  learning  objects  are  used  may  not  reflect  these 
principles.  This  disparity  may  be  more  extensive  than  what  we  can  observe  in  classroom  practice  or  through  what  is 
captured  through  digital  learning  systems  that  can  track  and  document  learner  actions.  How  often  or  frequently  a 
student  posts  messages  or  access  information  cannot  tell  us  about  their  level  of  thinking,  therefore,  we  should  design 
around  principles  that  are  most  likely  to  result  in  deeper  learning.  Deeper  learning  has  been  conceptualized  in 
learner-centered  principles;  although  there  are  several  approaches  to  articulating  such  principles,  the  author’s  uses 
those  principles  put  forth  by  Carmean  and  Haefner  (2002)  because  these  are  specifically  articulated  for  the 
application  of  technology  in  higher  education.  According  learner-centered  principles,  deeper  learning  occurs  when 
the  learning  experience  is  social,  active,  contextual,  engaging,  and  student-owned.  Active  learning  focuses  on  real 
world  problems  through  which  learners  practice  and  receive  feedback  and  reinforcement  from  peers  and  experts. 
Learning  that  is  contextual  must  focus  on  the  learner  rather  than  the  content.  A content-driven  approach  doesn’t 
readily  allow  for  learner  differences  or  modifications,  but  learning  objects  can  modify  content  so  that  is 
contextualized  for  the  individual  (i.e.,  geographical  location,  system  interface,  prior  knowledge,  etc.).  Deeper 
learning  requires  consideration  of  the  learner’s  learning  environment,  ways  of  learning,  as  well  as  experiences  in  the 
world  (e.g.  level  of  reading  ability,  language,  text,  vs.  visual,  etc.).  Learning  that  is  social  requires  feedback  and 
interaction  between  learners  and  instructor  and,  in  the  case  of  learning  objects,  feedback  may  be  also  situated  in  the 
technology.  Engaging  learning  has  to  be  designed  to  appeal  to  the  learner’s  preferences  and  experiences  but  also  to 
their  order  to  motivate  and  challenge.  Individualized  learning  provides  the  learner  multiple  learning  paths,  multiple 
representations  of  content,  multiple  strategies,  and  multiple  options  for  engagement  and  motivation  to  meet  one 
objective.  If  the  learner  is  to  have  ownership  over  what  they  are  learning,  there  must  be  some  aspect  of  self- 
determination  and  control  that  allows  them  to  examine  and  evaluate  unfamiliar  content  and  process,  and  to  be 
recognized  for  their  actions  or  productions.  Is  this  way  the  leaner  must  make  decisions  and  come  to  conclusions, 
supporting  independent  thinking  and  reflection. 

Embedding  deeper  learning  principles  into  learning  objects  by  actively  engaging  the  learner  and  providing 
them  ownership  of  knowledge  they  have  produced  ensures  that  learning  transcends  the  classroom.  In  order  to  apply 
these  principles  to  learning  objects,  we  must  consciously  consider  the  entire  learning  environment  and  how  we 
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articulate  the  processes  within  them.  For  example,  Laurillard  and  McAndrew  (2003)  propose  the  design  of  generic 
learning  activities  that  shift  teaching  from  a transmission  model  to  a construction  model.  Their  conversational 
framework  is  an  iterative  process  requires  the  learner  to  engage,  act,  and  reflect  upon  what  they  know  and  how  they 
come  to  learn  through  the  production  of  learning  objects.  The  learner  is  deeply  engaged  through  a process  of 
determining  and  interacting  with  the  content  and  others. 

Diaz  and  McGee  (2005)  have  identified  a contribution  pedagogy’  model  of  knowledge  management  utilized 
by  higher  education  and  evidenced  in  social  networks  that  engages  the  learner  in  constructing  content  thereby 
contributing  to  the  knowledge  base  of  the  course,  if  not  the  institution  or  social  network  at  large.  The  contribution 
pedagogy  model  is  based  on  certain  assumptions:  knowledge  is  generated  through  a learning  community  (such  as  a 
class);  content  that  is  generated  by  learners  has  multiple  benefits  (to  the  instructor,  peers,  and  others);  there  is  an 
increasing  shift  from  transmission  to  construction  of  knowledge;  and,  this  model  is  primarily  pedagogical.  The 
contribution  pedagogy  model  is  typically  course-based  and  enacted  through  a system,  such  as  an  L/CMS  or  other 
collaborative  systems.  Such  systems  are  varied,  for  example  Elgg1  Collaborative  Online  Reporting  (COR)2,  Online 
Student  Collected  and  Reported  Annotated  Resources  (OSACR)3,  and  Wikipedia4  are  all  collaborative  in  nature  and 
allow  users  to  contribute  what  they  know  as  an  object.  Increasingly  there  is  evidence  of  peer-to-peer  forms  of 
knowledge  sharing  that  are  related  to  learning  and  in  the  form  of  objects  such  as  videojug5  and  Second  Life6. 

The  trend  in  knowledge  management  through  organized  sharing  reflects  the  shift  toward  a learning  object 
pedagogy  in  which  learners  not  only  learn  from  experience  by  participating  in  the  generation  of  the  object  but  also 
by  contributing  to  the  learning  of  others  through  object  development  and  re-use.  Collis  and  Strijker  (2003)  suggest 
that  by  having  learners  generate  learning  objects,  and  contribute  to  a course  repository  that  grows  with  each  offering 
of  the  course,  the  burden  of  producing  objects  is  shifted  away  from  the  institution  and  the  instructional  process.  This 
results  in  a variety  of  benefits:  time  is  saved  for  the  instructor  or  content-generator,  resources  are  designed  by  the 
population  for  which  they  are  intended  by  providing  a locally  better  “fit”  with  the  intended  audience,  learners  can 
contribute  and  revise  objects  over  time  by  updating  content  or  presentation,  and  the  tacit  knowledge  of  the  learner  is 
transparent  and  can  be  shared  or  studied  by  the  institution  (Collis  & Winnips,  2002).  Such  an  approach  extends  on 
deeper  learning  where  the  learners  are  sharing,  doing  authentic  tasks  with  others,  and  contributing  through 
interactions  and  productions  (Collis  & Moonen,  2005).  By  allowing  the  learning  to  contribute  self-generated 
materials  that  are  publicly  available  and  used  by  learners  and  instructors  alike,  contribution  pedagogy  is  enacted. 

Project  Overview 

To  truly  support  deeper  learning,  learning  objects  must  be  designed  to  consider  all  facets  of  deeper  learning 
principles.  Interactivity  increases  the  active  engagement  of  the  learner  and  adaptive  content  can  address  the  learner’s 
context.  The  social  aspect  of  deeper  learning  has  not  been  clearly  demonstrated  in  standalone  objects  unless 
interactive  functions  are  attached  or  associated,  such  in  a learning/course  management  system  (L/CMS).  One  aspect 
of  deeper  learning  that  is  most  challenging  to  achieve  is  ownership.  There  do  exist  objects  that  aggregate 
performance  items  that  are  saved  or  stored  and  result  in  something  that  documents  the  learner’s  abilities.  There  also 
exist  learning  objects  that  are  generative  in  the  sense  that  they  allow  the  user  to  produce  something  of  their  own 
creation:  a product  ‘owned’  and  taken  away  by  the  learner.  However,  ownership  implies  more  than  production  and 
performance;  it  requires  that  the  learner  is  acknowledged  for  their  thinking,  effort,  performance,  or  production.  It  is 
the  author’s  contention  that  learning  objects  should  embody  the  notion  of  contribution  pedagogy  by  inviting  users  to 
contribute  content,  thus  engaging  the  learner  in  the  process  and  content  of  learning  rather  than  just  adapting  to  the 
learners  predispositions,  the  content  delivered  through  the  instructional  model  that  the  object  embodies  could  come 
from  any  discipline.  It  is  the  proof  of  the  instructional  concept  that  is  illustrated  in  this  paper. 

The  content  of  the  learning  object  designed  to  illustrate  how  contribution  pedagogy  can  be  enacted  comes 
from  visual  culture  (see  Elkins,  2003;  Mirzoeff,  1999;  Mulvey,  1999)  chosen  because  the  concepts  from  this  area 
can  be  applied  in  multiple  disciplines  (communication,  business,  education,  fine  arts)  and  in  multiple  contexts 
(continuing  education,  higher  education,  secondary  education,  training).  Entitled  Eye-to-Eye  this  learning  object  was 
designed  in  Flash™  using  XML  with  a database  that  allowed  random  access  to  learner-generated  objects. 


1 http://elgg.net/ 

2 http://cedir.uow.edu.au/deploy/cortex_demo/ 

3 http://cedir.uow.edu.au/deploy/iact_demo/ 

4 http://en.wikipedia.org/ 

5 http://www.videojug.com/ 

6 http://secondlife.com 
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The  object  design  draws  from  several  instructional  strategies.  First,  as  a high  level  organizational  strategy, 
IMS  learning  design  specifications  (IMS,  n.d.)  were  followed,  using  a method  that  incorporates  play  and  act  with  a 
role  as  author  to  complete  a cycle  of  learning  experiences,  see  Figure  1.  Second,  the  tutorial  (IMS  method)  and 
practice  IMS  play)  components  were  designed  according  to  Shepard’s  (2003)  Learning  Object  Design  Assistant 
(LODA).  Shepard  describes  three  types  of  objects:  information,  tutorial,  and  practice.  The  Eye-to-Eye  project 
incorporates  all  three  types,  treating  each  as  a shared  content  object  (SCO)  that  when  aggregated  provide  a complete 
learning  experience.  However,  each  SCO  is  designed  so  that  it  can  be  used  independently. 


Figure  1.  IMS  Learning  Design  Overview 


An  instructional  analysis  resulted  in  the  development  of  a tutorial,  practice  area,  and  then  a design  area  in  which  the 
user  can  create  (using  images  from  a digital  repository)  a digital  product  that  becomes  part  of  a database  and 
subsequently  integrated  into  the  tutorial  component  of  the  object  see  Figure  2.  Associate  throughout  the  first  two 
SCOs  is  a glossary  that  defines  key  terms  relating  to  the  power  of  visual  point  of  view  and  how  we  can  use  it  to 
interpret  and  convey  meaning.  Users  can  enter  the  tutorial  and  proceed  through  an  overview  of  theory  and  concepts 
and  practice  what  they  have  learned.  Users  also  have  the  option  of  going  directly  to  the  third  SCO  in  which  the  sure 
can  create  their  own  SCO.  The  third  SCO  is  supported  with  an  asset  that  explains  how  to  use  the  interface,  as  well  as 
an  asset  that  provides  guidelines  for  creating  a powerful  story. 
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Figure  2.  Overview  of  Eye-to-Eye  SCOs 


When  the  user  enters  the  object  he  or  she  is  given  the  choice  of  entering  the  tutorial  or  the  practice  area,  see  Figure 
3.  The  tutorial  SCO  reviews  visual  point  of  view  principles  while  allowing  the  user  to  explore  key  concepts.  The 
practice  SCO  allows  the  user  to  make  decisions  and  enter  information  that  is  complied  and  re-presented  throughout 
the  practice  area.  In  this  way,  the  user  is  able  to  make  sense  of  what  they  have  learned,  and  tentatively  apply 
(practice)  their  new  knowledge. 
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The  final  SCO  is  where  the  user  is  able  to  act  an  author  when  they  create  their  own  object.  They  user  must  create  a 
user  name  and  password  that  is  then  associated  with  the  object  that  is  created.  Once  in  the  SCO,  the  user  selects  a 
theme,  selects  and  orders  images,  and  adds  text  subtitles  to  the  images.  The  ultimate  goal  is  to  allow  users  to  upload 
their  own  images  and  add  audio,  however,  institutional  restrictions  have  not  permitted  this  option  so  far.  Objects  that 
users  create  are  stored  in  a database  until  randomly  called  up  in  the  tutorial. 

Unlike  course-based  objects,  this  project  was  designed  to  be  used  either  within  a traditional  course  or  as  a 
standalone  object.  In  this  way,  multiple  perspectives  from  a variety  of  disciplines  and  life  experiences  can  be  saved 
and  reflected  in  the  content  of  the  tutorial. 


Conclusions 

The  trend  toward  open  knowledge  suggests  that  student  contributions  to  course  content  will  continue  to  be 
seen  in  higher  education  and  given  the  concern  with  identifying  intellectual  property  rights  (such  as  The  Digital 
Object  Identifier7  (DOI®),  Digital  Rights  Management8  (DRM),  and  the  increasing  number  of  digital  repositories. 
Shifts  toward  an  open  knowledge  economy  appear  inevitable  (Norris,  Mason,  & Lefrere,  2003),  and  efforts  such  as 
the  Creative  Commons9  project  are  evidence  that  there  is  a demand  for  strategies  that  support  the  exchange  of 
individually  (rather  than  institutionally)  generated  content.  Johnson  (2003)  describes  five  markets  in  the  learning 
object  economy:  proprietary  (e.g.,  private  training  venues),  commercial  (e.g.  publishers),  free  (e.g.  MERLOT), 
shared  (e.g.  consortia),  and  peer-to-peer  (between  institutions  or  individuals).  A pedagogy  that  embraces  trends  in 
the  economic  market  and  social  network  for  sharing  and  contribution  is  appealing  to  the  learner  as  well  as  to  the 
institution  that  will  benefit  from  engaged  and  productive  learners. 
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Assessing  What  Really  Matters:  Rubrics  Can  Improve  Student  Achievement 
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Educators  look  for  tools  and  motivational  processes  that  will  enhance  learners’  acquisition  of  knowledge, 
skills,  and  attitudes,  as  well  as  improve  achievement  of  academic  objectives.  Scoring  rubrics  are  highly  touted  in 
books  and  practitioner  publications  and  are  widely  used  at  many  academic  levels  as  a teacher’s  tool  to  expedite  the 
grading  process  (Alter  & McTighe,  2001;  Goodrich,  1996;  Popham,  1997;  Stevens  & Levi,  2005;  Wiggins,  1998).  A 
rubric  is  typically  1-2-pages  and  has  three  essential  features:  evaluative  criteria,  gradations  of  quality  definitions, 
and  a scoring  strategy  (Stevens  & Levi,  2005). 

A well  constructed  rubric  sets  learning  goals,  guidelines,  and  levels  of  standards  for  performance  of  a 
specific  assignment  (Andrade,  2000;  Alter  & McTighe,  2001).  It  is  typically  used  for  evaluating  what  really  matters: 
relatively  complex,  open-ended  tasks,  projects,  or  assignments  that  require  integration  of  multiple  skills  and  criteria 
(Arter  & McTighe,  2001).  These  types  of  works  allow  students  to  develop  the  knowledge  and  abilities  that  educators 
look  for  and  that  society  values  in  workers  (i.e.,  critical  thinking,  problem-solving,  creativity).  However,  a rubric  is 
more  than  just  an  evaluation  tool.  It  has  instructional  benefits  that  are  potentially  strengthened  when  instructional 
guidance  on  the  use  of  the  rubric  is  offered  students,  such  as  helping  to  improve  student  motivation,  support  learning 
during  the  process,  and  raise  achievement  (Andrade,  2000;  Stevens  & Levi,  2005). 

Instructional  Benefits  of  Rubrics 

The  instructional  benefits  of  using  rubrics  go  beyond  teacher  efficiency  for  grading,  to  clarifying  and 
communicating  the  learning  objectives,  and  promotion  of  principles  of  equity  and  fairness.  As  an  instructional  tool, 
rubrics  make  teacher’s  expectations  very  clear  (Andrade  & Du,  2005);  provide  meaningful,  informative  feedback  to 
learners  about  their  strengths  and  areas  in  need  of  improvement  (Andrade,  2000);  communicate  the  goals  to  students 
and  help  them  plan  an  approach  to  an  assignment  (Andrade  & Du,  2005);  support  the  development  of  critical 
thinking,  skills,  and  understanding  (Andrade,  2001);  and  help  students  develop  personal  habits  of  self-assessment 
(Saddler  & Andrade,  2004;  Stevens  & Levi,  2005),  such  as  checking  their  work,  revising,  and  reflecting  on  feedback 
(Andrade  & Du,  2005). 

Rubrics  allow  instructors  to  communicate  the  importance  of  some  academic  skills  over  others.  The  criteria 
for  rubrics  are  assigned  a value,  points  or  percentage,  according  to  the  importance  of  those  criteria  on  the  final 
product.  By  including  a scoring  value  that  makes  it  clear  that  those  components  that  relate  to  a specific  outcome 
(Stevens  & Levi,  2005).  Meaningful  feedback  on  student  performance  has  been  found  to  promote  higher  levels  of 
academic  achievement  for  all  grade  levels  and  is  most  effective  when  it  contains  as  much  detailed,  specific 
information  as  possible  (Brinko,  1993).  Students  benefit  from  the  use  of  clearly  defined  performance  criteria  that 
encourage  a shared  vocabulary  and  a clearer  understanding  of  the  important  dimensions  of  quality  performance.  The 
rubric  offers  information  about  the  highest  level  of  achievement  possible,  the  strengths  of  their  work  and  areas  for 
improvement.  As  a result,  the  student  can  read  the  higher-level  description  as  self-assessment  guidelines  for 
improvement. 

Used  in  conjunction  with  sound  instructional  practice,  rubrics  can  contribute  to  students’  development  of 
higher  order  thinking  skills  such  as,  analysis,  evaluation,  and  creativity  (Huba  & Freed,  2000).  In  addition,  rubrics 
encourage  critical  thinking  by  showing  expected  levels  of  performance  or  achievement  and  inspiring  students  to 
develop  the  reflective  practice  of  self-assessment  and  self-improvement.  In  a study  of  8th  graders  writing  a 
persuasive  essay,  Andrade  (2001)  proposed  that  thinking-centered  rubrics  helped  students  think  more  deeply.  To 
promote  critical  thinking,  a criterion  was  included  in  the  rubric  that  prompted  the  student  to  anticipate  the  other  side 
of  the  argument  and  prepare  an  explanation  in  defense  against  it.  Students  without  the  rubric  did  not  consider  the 
counterpoint  arguments  against  their  claim. 
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Despite  the  noted  benefits  of  using  rubrics,  for  many  educators,  rubrics  don’t  live  up  to  their  promise  as 
effective  guides  for  both  teachers  and  students  (Popham,  1997).  Some  teachers  perceive  rubrics  are  difficult  to 
construct  and  don’t  adequately  capture  their  intentions  for  evaluating  student  performance.  In  practice,  teachers  who 
use  rubrics  often  present  it  to  the  learner  without  meaningful  instructional  guidance  or  discussion  about  its  purpose 
or  effective  use  for  completing  the  assignment  (Alter  & McTighe,  2001).  As  a result,  students  don’t  benefit  from  the 
rubric  to  improve  performance  and  educators  feel  their  effort  to  create  it  was  wasted.  Given  the  mixed  evidence  on 
rubric  use,  what’s  a good  strategy  for  using  it  to  promote  the  types  of  outcomes  the  teacher  deems  important? 

Design  an  Effective  Rubric 

A best  practice  for  effective  rubric  use  is  to  begin  with  a well-designed  rubric.  It  is  challenging  work  to 
create  a rubric,  but  the  effort  doesn’t  have  to  be  arduous.  Arter  and  McTighe  (2001)  provide  clear  and  extensive 
guidance  for  designing  effective  rubrics.  Begin  by  collecting  examples  of  student  performance  that  demonstrate  the 
skill  or  behavior  under  consideration.  Next,  sort  the  student  work  into  groups  and  write  down  your  rationale.  Group 
labels  could  be  as  simple  as  strong,  middle,  and  weak.  Third,  examine  your  list  of  rationale  and  cluster  the  reasons 
into  “traits”  or  important  dimensions  of  performance.  Next,  write  a definition  of  each  trait  that  describes  what  the 
trait  is  about,  not  what  good  performance  looks  like.  Fifth,  find  multiple  examples  of  student  work  that  exemplifies 
the  strong,  weak,  and  middle  performance  on  each  trait.  Last,  conduct  formative  evaluations  of  your  rubric  and 
refine  the  descriptions  so  they  communicate  better.  At  this  point  you  should  use  a metarubric,  a rubric  for  rubrics,  to 
evaluate  the  quality  of  your  rubric. 

There  is  no  single  right  way  to  design  and  use  a rubric.  Another  method  described  by  Stevens  and  Levi 
(2005)  includes  four  key  stages:  Reflecting,  Listing,  Grouping  and  Labeling,  and  Application.  Stage  1:  Reflecting 
involves  thinking  about  the  desired  learning  outcomes  of  the  assignment  and  what  your  expectations  are.  Stage  2: 
Listing  focuses  on  details  of  the  assignment  and  the  learning  objectives.  In  Stage  3:  Grouping  and  Labeling,  you 
organize  your  reflections  into  groups  of  similar  expectations  that  will  likely  become  the  dimensions  of  the  rubric. 
Stage  4:  Application  is  where  you  apply  the  dimensions  and  descriptions  from  Stage  3 to  the  grid  format  of  the 
rubric. 


These  two  process  examples  provide  guidance  for  what  to  do  when  designing  effective  rubrics,  but  there 
are  still  more  methods  for  how  to  design  rubrics.  The  continuum  of  methods  ranges  from  primarily  instructor-driven 
to  primarily  student-driven  (Stevens  & Levi,  2005).  While  it  may  initially  seem  risky  to  involve  students  in  rubric 
construction,  it  can  be  very  effective  and  beneficial  for  both  students  and  the  instructor.  Students  become 
stakeholders  in  the  educational  process  creating  greater  student  buy-in  to  the  assessment  standards  (Whittaker, 
Salend,  & Duhaney,  2001).  Student  participation  helps  prevent  misunderstandings  and  misinterpretations  of  the 
rubric  before  they  affect  student  work  (Stevens  & Levi,  2001). 

Provide  Instructional  Guidance 

Instructional  guidance  on  the  use  of  a rubric  to  complete  a task  has  been  shown  to  increase  the  overall 
utility  and  effectiveness  of  the  rubric  (Andrade,  2000,  2001;  Andrade  & Du,  2005;  Stevens  & Levi,  2005).  One  good 
instructional  practice  that  promotes  critical  thinking  and  effective  use  of  the  rubric  is  classroom  discussion  of  the 
rubric  prior  to  the  students  beginning  the  assignment.  In  a focus  group  study  of  undergraduate  students,  Andrade  and 
Du  (2005)  found  that  undergraduate  students’  perception  of  improvements  in  the  quality  of  their  writing  (i.e., 
student  achievement)  was  attributed  to  understanding  and  using  a rubric.  Another  study  (Andrade,  2001)  examined 
the  effects  of  rubrics  on  8th  graders  writing  skills  and  found  that  simply  handing  out  and  explaining  a rubric  seemed 
to  help  students  improve  their  writing. 

Andrade  (2000)  notes  that  while  research  studies  present  evidence  for  a link  between  rubrics  and  student 
learning,  the  mechanisms  behind  the  achievement  advantage  provided  by  rubrics  is  not  known.  Research  has  shown 
that  if  a rubric  is  used  as  part  of  the  instructional  strategy  when  the  assignment  is  given  to  students,  it  is  likely  to 
serve  its  purposes  of  informing  students  of  the  standards  of  performance  and  supporting  their  achievement 
(Andrade,  2001).  When  learners  better  understand  performance  standards  for  the  task,  it  is  likely  they  will  be  able  to 
compare  their  performance  with  the  standard  (Andrade  & Du,  2005).  Andrade  discovered  rubrics  are  not  completely 
self-explanatory,  so  students  need  help  in  understanding  rubrics  and  their  use.  When  students  where  given  a rubric 
without  explanation,  “The  more  motivated  students  anguished  over  what  to  do  with  it,  and  the  less  motivated  filed  it 
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in  their  notebooks  and  promptly  forgot  about  it"  (Andrade,  2005,  p.  29).  The  rubric  and  standards  for  success  should 
not  be  a mystery  to  students,  rather  when  students  know  the  criteria  in  advance  of  their  performance,  they  are 
provided  with  clear  goals  for  their  work.  In  the  same  way,  when  students  have  access  to  performance  criteria  and 
scoring  guides,  they  have  the  opportunity  to  self-assess  and  improve  their  work  as  it  is  developed  (Wiggins,  1998). 
This  method  can  enhance  the  quality  of  student  learning  and  performance,  not  just  evaluate  it  (Alter  & McTighe, 
2001).  Some  of  the  rubric’s  criteria  break  down  the  components  of  critical  thinking  in  an  explicit  manner,  while  the 
descriptions  of  the  criteria  explicitly  lay  out  the  requirements  of  how  the  student’s  execution  of  the  criteria  can  be 
critically  analyzed.  Critical  analysis  of  the  rubric’s  criteria  may  be  so  basic  to  instructors  creating  the  rubric  that  they 
are  left  implicit  in  the  assignment  and  so  may  be  overlooked  by  the  students  until  the  assignment  is  complete. 

The  instructional  strategy  of  discussing  the  rubric  and  communicating  its  purpose  in  advance  makes  the 
instructor’s  implicit  expectations  explicit.  Interviews  with  undergraduates  in  an  educational  psychology  course 
regarding  rubric  use  revealed  they  liked  having  the  rubric  in  advance  so  that  they  knew  “what’s  expected,”  and 
contrasted  it  with  the  “guessing  game"  they  felt  they  had  to  play  when  instructors  did  not  provide  a rubric  (Andrade 
& Du,  2005).  In  addition,  instruction  on  rubric  use  models,  in  reverse,  the  criteria  by  which  the  work  will  be  graded 
(Stevens  & Levi,  2005)  and  supports  the  development  of  academic  skills  and  understanding  pertinent  to  achieving  a 
level  of  quality  for  the  assignment  (Andrade,  2000). 

Rubrics  Promote  Motivation 

Even  with  the  combined  benefits  of  good  design  and  instructional  guidance,  any  tool  would  have  minimal 
effect  if  the  user  were  not  motivated  to  use  it.  Students’  motivation,  namely  self-efficacy,  for  accomplishing  a task 
may  be  increased  when  using  a rubric,  resulting  in  higher  student  achievement.  Self-efficacy  is  a belief  system 
proposed  by  Bandura  (1986,  1997)  and  is  defined  as  a person’s  judgment  of  their  ability  to  plan  and  carryout  the 
actions  required  to  achieve  specific  types  of  performances.  Self-efficacy  is  believed  to  be  an  important  variable  in 
understanding  students’  behaviors  in  educational  contexts  (Schunk,  1989)  and  to  significantly  influence  students’ 
motivation  to  learn  and  for  skillful  performance  (Pintrich  & Schunk,  2002;  Schunk,  1989).  A learner’s  self-efficacy 
beliefs  are  a key  factor  in  motivation  for  accomplishing  a specific  task  (Pintrich  & Schunk,  2002),  such  as  arithmetic 
division  problems  (Schunk,  1981)  and  writing  performance  (Meier,  McCarthy,  & Schmeck,  1984).  According  to 
Bandura  (1997)  self-efficacy  is  a generative  characteristic  and  therefore  can  change  depending  on  the  task.  At  the 
start  of  a task,  students  have  different  beliefs  about  their  capabilities  to  acquire  knowledge,  perform  skills,  master 
the  materials,  and  so  on  (Schunk,  1989).  This  initial  sense  of  self-efficacy  varies  as  a function  of  previous 
experience  and  personal  characteristics  and  therefore  can  be  promoted.  Interventions  that  have  been  found  to 
enhance  learners’  self-efficacy  include:  skills  for  self-evaluation,  self-monitoring,  and  experiencing  progress 
towards  set  goals  (Schunk  & Ertmer,  2000);  ability  to  synthesize  and  analyze  content  (Meier,  McCarthy,  & 

Schmeck,  1984);  setting  goals  (Schunk  & Ertmer,  2000;  Zimmerman,  Bandura,  & Martinez-Pons,  1992);  receiving 
rewards  for  meeting  set  goals  (Schunk,  1984);  possessing  knowledge  of  what  to  do,  perceiving  a challenge  worth 
accomplishing,  and  receiving  performance  feedback  about  progress  (Pintrich  & Schunk,  2002);  and  the  teaching  of 
effective  learning  strategies  that  will  help  them  accomplish  the  set  goals  (Corno  & Mandinach,  1983;  Schunk, 
1987b).  See  Appendix  A:  Concept  Map  of  Relationship  of  Rubric,  Instruction,  Self-Efficacy,  and  Academic 
Achievement. 

Self-efficacy  and  Rubrics 

Rubrics  have  the  potential  to  promote  students’  efficacy  for  a task  by  serving  as  the  mechanism  for  the  self- 
efficacy  enhancement  interventions  noted  previously.  Although  a review  of  the  literature  has  not  revealed  an 
empirical  research  study  that  directly  correlates  rubrics  and  self-efficacy,  the  literature  does  yield  a body  of  research 
about  three  correlated  concept  pairs:  rubrics  and  self-evaluation  (or  self-assessment);  self-evaluation  and  academic 
achievement;  and  academic  achievement  and  self-efficacy.  Self-evaluation  is  the  key  construct  that  ties  rubrics  to 
self-efficacy.  Rubrics  can  be  used  as  a self-evaluation  tool  for  learners  (Stevens  & Levi,  2005)  and  self-evaluation  is 
one  of  the  factors  affecting  self-efficacy  (Zimmerman  & Martinez-Pons,  1990).  The  learner  can  self-assess  his  or  her 
performance,  in  part,  on  feedback  from  others  (Bandura,  1997),  themselves  (Schunk,  2003),  or  materials  (Pintrich  & 
Schunk,  2002),  like  rubrics. 

Zimmerman  and  Schunk  (2001)  found  in  multiple  research  studies  that  self-assessment  activities  can 
promote  achievement  and  inspire  learning  behaviors  typically  associated  with  academic  success,  such  as  goal 
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setting,  self-monitoring,  and  revision.  A well-designed  rubric  makes  self-assessment  easier,  particularly  when  used 
as  a “goal-setting  guide,”  and  shared  with  students  prior  to  starting  a task  to  help  them  understand  the  nature  of  the 
levels  of  achievement  goals  and  performance  expectations  for  each  level  (Andrade  & Du,  2005;  Alter  & McTighe, 
2001). 


The  process  of  self-evaluation  of  performance  can  be  supported  by  the  use  of  a rubric  that  specifically 
describes  the  criteria  against  which  the  learner  compares  their  performance.  Another  link  between  self-efficacy  and 
rubrics  is  when  learners  know  they  are  making  progress  and  success  towards  a set  goal  based  on  information  of 
performance  levels  detailed  in  the  rubric.  The  learner  can  set  goals  for  task  performance  based  on  the  criteria 
explicitly  stated  in  the  rubric  or  derived  from  it.  The  criteria,  as  well  as  the  organization  of  the  rubric,  can  inform  the 
learner  of  the  processes  and  procedures  for  completing  the  task  and  help  them  determine  “what  counts”.  Rubrics  can 
be  linked  to  self-efficacy  via  feedback  the  learner  receives  from  the  rubric  while  completing  the  task  (Pintrich  & 
Schunk,  2002).  Determining  one’s  self-efficacy  requires  information  about  one’s  performance,  which  can  be 
evaluated  with  the  standards  set  in  the  rubric.  Self-efficacy  can  be  increased  when  learners  are  taught  effective 
learning  strategies.  Instruction  on  using  a rubric  can  be  considered  a learning  strategy  and  if  taught  as  guided 
practice,  is  likely  to  increase  self-efficacy,  and  subsequently  academic  achievement. 

Conclusion 

Assessing  what  matters,  such  as  complex,  open-ended  tasks  that  involve  critical  thinking,  problem  solving, 
or  creativity  require  a good  description  of  the  traits  and  qualities  that  comprise  successful  performance.  Rubrics 
provide  a structure  for  organizing  the  list  of  traits  by  levels  of  qualities  for  instructor  and  student  use.  Developing  a 
rubric  can  be  challenging,  but  teachers  can  follow  a number  of  well-documented  procedures  described  in  the 
literature  to  mitigate  the  process. 

By  itself,  a rubric  offers  instructional  benefits,  like  clarifying  the  teacher’s  expectations  and  providing  the 
student  with  meaningful  feedback.  When  coupled  with  instructional  guidance,  a rubric  has  the  potential  to  increase 
student  self-efficacy  and  achievement  for  the  task.  Self-efficacy  is  the  belief  a student  has  that  they  can  accomplish  a 
task  and  motivates  them  to  apply  more  effort.  The  rubric  supports  self-efficacy  by  offering  the  student  a way  to  self- 
evaluate  their  work,  get  meaningful  feedback,  and  determine  their  progress  toward  the  goal  of  the  task.  The  result 
could  lead  to  higher  student  achievement.  Although  the  outcomes  of  rubric  use  are  by  no  means  guaranteed,  the 
potential  for  improved  student  achievement  is  tantalizing. 
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Be  a Scientist,  Do  Practical  Science: 

Teachers  Explore  Marine  Biotechnology  and  Bioinformatics 


Steven  J.  McGriff 
San  Jose  State  University 

Introduction 

Science  teachers  in  California  are  required  to  teach  the  state  standard  for  inquiry-based  science,  molecular 
biology,  genetics,  and  the  role  of  technology  and  computer  analysis  in  the  study  of  biology.  However,  the  field  is 
changing  so  rapidly  many  teachers  lack  a clear  understanding  of  how  to  do  science  using  current  technology  and 
how  to  make  the  complexity  of  science  relevant  to  their  students.  A three-week  summer  workshop,  Marine 
Biotechnology’  and  Bioinformatics  for  Teachers,  was  held  in  July  2006,  at  the  California  State  University,  Monterey 
Bay  campus  in  California,  where  13  middle-  and  high-school  science  teachers  were  paid  a stipend  and  immersed  in 
“doing  science  like  scientists  do.”  The  teachers  were  from  the  San  Francisco  Bay  Area,  the  California  Central  Coast, 
Southern  California,  and  one  from  Georgia. 

The  workshop  was  a comprehensive  project  sponsored  by  the  Information  Technology  Experiences  for 
Students  and  Teachers  (ITEST)  program,  established  by  the  National  Science  Foundation  (NSF)  to  address  the 
looming  shortage  of  technology  workers  in  the  United  States  (ITEST,  2006).  Three  principle  investigators  and 
project  staff  from  CSU  Monterey  Bay,  Moss  Landing  Marine  Labs,  and  San  Jose  State  University  collaborated  to 
develop  the  workshop. 

The  workshop  was  situated  in  a problem-based  learning  context  within  the  field  of  marine  biotechnology 
and  bioinformatics.  The  teachers  used  current  research  methods  for  DNA  extraction  and  analysis  of  mussels 
searching  for  an  invasive  species.  The  workshop  was  designed  with  instructional  strategies  for  rapid  integration  of 
new  knowledge  and  best  practices  for  transforming  high-level,  new  science  knowledge  into  lesson  plans  for 
students.  Overall,  the  teachers  had  a very  positive  response  to  doing  rigorous  science  that  took  them  out  in  the  field 
and  back  into  the  wet  lab.  Recommendations  and  future  directions  for  the  next  two  workshops  are  discussed. 

Outdated  Standards 

California  education  standards  for  science  content  include  planning  and  conducting  experimental 
investigations  in  cell  biology  and  genetics,  including  DNA  sequencing.  For  many  experienced  teachers  the 
technology  and  research  methods  have  evolved  since  they  were  formally  trained  and  the  state  standards  have  not 
kept  pace.  Many  other  science  teachers  have  not  had  an  in-depth  exposure  to  the  practice  of  scientific  inquiry.  This 
workshop  was  designed  to  begin  a process  for  addressing  the  lack  of  current  science  standards  involving  technology. 

California  and  National  Standards 

The  California  science  standards  include  one  high  school  standard  for  biotechnology,  listed  under 
Biology/Life  Sciences,  Genetics,  and  referenced  as  genetic  engineering.  There  is  no  reference  found  in  the 
California  science  or  technology  standards  for  bioinformatics  (California  State  Board  of  Education,  1998).  The 
subject  matter  experts  for  this  workshop  defined  biotechnology  as  the  use  of  biological  processes  to  solve  problems 
or  make  useful  products.  Bioinformatics  is  defined  as  the  use  of  computers  to  store,  retrieve  and  analyze  biological 
data.  Further  searching  for  information  technology  yielded  no  result  in  the  California  standards.  The  national  science 
education  standards  do  not  have  a standard  involving  biotechnology,  bioinformatics,  or  information  technology  and 
the  word  “technology”  is  not  found  in  the  276-page  document  (1996). 

Marine  Biotechnology  and  Bioinformatics  for  Teachers  Workshop 

Learning  the  nature  of  scientific  inquiry  is  better  suited  for  an  extensive,  three-week,  immersive,  hands-on 
workshop,  where  teacher  participants  can  really  get  into  doing  what  scientists  do  while  conducting  research.  The 
goal  of  the  workshop  was  to  develop  the  teachers’  ability  to  assimilate  high-level,  scientific,  and  technological 
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information  and  procedures,  then  break  it  down  into  teachable  chunks  suitable  for  the  middle-  and  high-school 
students  in  their  classrooms. 

Marine  biotechnology  was  used  to  gather  data  that  could  be  analyzed  using  bioinformatics  techniques.  The 
field  of  marine  biotechnology  was  chosen  for  the  workshop  because  two  principle  investigators  had  both  specialized 
in  marine  biology.  Information  technology  was  integrated  with  marine  biology  in  the  form  of  bioinformatics.  The 
workshop  theme  was  based  on  an  authentic  biotechnology  problem  for  scientific  inquiry:  identifying  an  invasive 
mussel  ( mytilus  galloprovincialis),  which  can  only  be  determined  by  examining  DNA  evidence  using  bioinformatics 
software. 

It  takes  time  to  do  science:  harvesting  mussels  from  the  local  waters  of  Monterey  Bay,  setting  up  wet  labs 
on  campus,  running  experiments,  and  analyzing  data  required  numerous  hours.  To  create  a successful  learning 
environment,  it  was  decided  to  use  an  immersive,  “hands-on,  mind-on”  model.  A majority  of  the  scheduled  learning 
experience  was  devoted  to  knowledge  acquisition  and  laboratory/field  science  practice  in  marine  biology  and 
biotechnology.  The  teachers  were  first  given  background  information  on  the  ecological  problem  this  invasive 
species  causes,  then  an  explanation  of  the  biotechnology  processes  that  would  enable  them  to  investigate  this 
problem.  The  second  morning,  the  teachers  collected  mussels  from  a local  pier  and  returned  to  the  wet  lab  where 
they  learned  a series  of  procedures  to  extract  and  process  the  DNA. 

Another  significant  portion  of  the  schedule  was  reserved  for  learning  to  use  bioinformatics  software  to 
manage  the  data  analysis  and  sequencing  of  the  DNA  extracted  by  each  participant  in  the  laboratory.  Early  in  the 
second  week,  the  teachers  used  their  DNA  sequences  that  had  been  processed  at  an  outside  laboratory  and  began  the 
analysis  with  software  such  as  chromas,  Blast,  and  Genbank. 

The  lesson  plan  development  process  was  integrated  throughout  the  first  two  weeks,  overlapping  the 
science  and  technology  topics,  and  culminated  in  the  lesson  implementation  in  the  third  week.  The  design  and 
development  of  the  lesson  plan  component  is  presented  in  more  detail  in  the  next  section.  More  information, 
resources,  and  materials  about  the  Marine  Biotechnology  and  Bioinformatics  for  Teachers  Workshop  can  be 
retrieved  from  http://marinebiotech.net. 

Workshop  Design  for  Rapid  Integration 

This  was  the  third  year  developing  the  workshop.  The  first  two  summer  workshops  were  designed  and 
developed  by  the  two  original  principle  investigators  based  on  their  expertise  with  marine  biotechnology  and 
bioinformatics.  A third  principle  investigator  with  expertise  in  science  education  and  instructional  technology  and  an 
instructional  designer  were  brought  onto  the  project  in  the  third  year.  The  designers’  primary  responsibilities  were  to 
incorporate  the  lessons  learned  and  develop  innovative  instructional  solutions  that  would  facilitate  the  transfer  and 
application  of  knowledge  to  the  teachers’  science  classroom.  State  standards  were  integrated  in  the  workshop 
curriculum  and  explicitly  taught  so  that  teacher  participants  can  more  easily  see  how  to  incorporate  them  into  their 
own  science  classrooms.  The  two  marine  scientist  investigators,  who  also  served  as  instructors,  used  the  latest 
science  methods  in  a technologically  advanced  laboratory  and  computer  lab. 

Pedagogical  issues  for  teaching  science  methods  were  addressed  by  having  teachers  immersed  in  a 
problem-based  learning  (PBL)  scenario  derived  from  current  marine  biotechnology  research  that  was  being 
conducted  at  nearby  Moss  Landing  Marine  Laboratories,  Moss  Landing,  CA.  Going  one  step  further,  teachers  were 
challenged  with  the  requirement  to  transform  their  new  knowledge  into  a lesson  plan  that  could  be  realistically 
taught  in  their  own  science  curriculum.  It  is  generally  accepted  that  a real  demonstration  of  mastery  is  to  teach 
someone.  The  NSF  grant  specified  that  teachers  should  be  lifelong  learners  and  willing  to  share  their  knowledge 
with  their  students.  The  issue  of  transfer  was  a high  priority  for  the  workshop  to  help  insure  that  the  teachers’ 
students  would  also  benefit  from  learning  about  current  science  methods  in  biotechnology  and  bioinformatics,  as 
well  as  careers  in  information  technology. 

Marine  biotechnology  and  bioinformatics  are  two  information  intensive  topics  that  could  have  easily  taken 
the  entire  three-week  workshop.  The  decision  to  also  teach  lesson  plan  development  created  a pedagogical 
challenge.  The  two  instructional  designers  worked  with  the  two  subject  matter  experts  to  develop  a full  15-day 
workshop  schedule  to  integrate  all  three  content  areas  and  balance  the  goals  of  the  workshop.  Instructional  strategies 
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and  planning  were  incorporated  to  support  and  facilitate  knowledge  and  skill  acquisition  of  the  specific  scientific 
methods  presented.  In  the  workshop  design,  lesson  plan  development  was  introduced  in  the  first  week  and 
overlapped  the  marine  biotechnology  and  bioinformatics  topics  to  allow  the  teachers  time  to  think  of  how  they 
would  create  a lesson  plan  based  on  what  they  were  learning.  It  was  expected  that  teachers  would  complete  their 
workshop  learning  experience  by  writing  a lesson  plan  that  would  be  delivered  in  their  own  classrooms  during  the 
academic  year.  In  addition,  teachers  were  expected  to  work  in  teams  to  develop  a lesson  and  teach  it  to  a group  of 
students  in  the  last  week  of  the  workshop.  To  facilitate  developing  lesson  plans  for  both  purposes,  half-day  sessions 
were  included  to  teach  problem-based  learning  with  WebQuests,  as  well  as  other  instructional  strategies.  The  two 
instructional  designers  taught  these  sessions  and  closely  supported  the  teacher  teams  during  the  development  phase. 

Although  science  teachers  working  within  the  Central  California  Coast  and  San  Francisco  Bay  Area 
regions  were  the  target  audience  for  this  summer  workshop,  the  second  important  beneficiary  of  the  workshop  were 
the  underrepresented  student  groups  whom  these  teachers  serve.  This  student  population,  which  comprises  low- 
income  Hispanics,  Latinos,  as  well  as  a significant  percentage  of  Asian  and  African-Americans,  does  not  advance 
into  careers  in  information  technology  or  STEM  careers  (science,  technology,  engineering,  and  math)  in  proportion 
with  their  numbers.  This  issue  causes  extreme  concern  in  light  of  the  multitude  of  well-paying  information 
technology  jobs  available  in  nearby  Silicon  Valley  and  the  entire  San  Francisco  Bay  Area.  To  meet  an  NSF  grant 
requirement  and  provide  teachers  the  opportunity  to  do  a formative  evaluation  of  their  lesson  plans,  20  local  middle- 
and  high-school  summer  students  were  invited  to  participate  in  the  workshop  during  the  last  week.  These  students 
were  a sample  of  the  student  population  targeted  by  NSF. 

Transparent  Instructional  Systems  Design 

Other  instructional  technology  related  interventions  used  during  the  workshop  were  transparent  to  the 
teachers.  For  example,  using  a basic  participatory,  team-planning  model,  teachers  formed  their  own  lesson  plan 
groups,  chose  a topic  from  the  workshop,  and  collaborated  with  the  other  three  groups  to  design  an  appropriate 
sequence  for  the  four  lessons.  The  lesson  implementation  process  was  a form  of  microteaching  and  included  the 
Japanese  Lesson  Plan  (Columbia)  protocol  (Chokshi,  Ertle,  Fernandez,  & Yoshida,  2001),  where  the  quality  of  the 
lesson  was  determined  by  observing  the  learners’  reactions.  Within  the  teacher  teams,  those  who  were  not  giving 
large  group  instruction  conducted  the  observations.  Further  instantiation  of  instructional  systems  design  occured  in 
the  development  of  templates  for  the  lesson  plan  format  and  other  instructional  materials  prepared  by  the  techers. 
The  lesson  plan  was  based  on  Bernie  Dodge’s  WebQuest  model  (Dodge,  2006)  and  incorporated  additional 
placeholders  for  standards,  assessment  of  learning  outcomes,  and  STEM  career  options.  The  students  who  were 
invited  to  participate  in  the  lesson  provided  each  team  with  immediate  feedback  as  evidenced  from  the  products  they 
produced  during  creative,  reflective  learning  activities  following  each  lesson.  In  addition,  the  students  then 
participated  in  a group  debriefing  and  anonymously  completed  a survey. 

Formative  evaluation,  a vital  component  of  the  instructional  design  process,  focused  on  the  teachers’  lesson 
plans  and  implementation: 

1.  Formative  evaluation  of  the  lesson  by  the  workshop  program  staff  confirmed  the  teachers  application  of 
knowledge 

2.  Written  feedback  from  the  student  participants  and  the  products  they  produced  after  the  lesson  helped  the 
teachers  know  the  degree  of  effectiveness  of  their  lesson 

3.  Peer  feedback  using  the  Japanese  Lesson  Plan  protocol  informed  the  teachers  of  how  well  their  lesson  was 
holding  the  students’  attention 

Best  Practices 

The  best  practices  for  teaching  complex  science  to  teachers  are  summarized  below: 

• Make  instructional  technology  strategies  transparent  to  participants 

• Use  project-based  learning  immersion 

• Allow  participatory  groups  to  share  ownership  of  the  learning  environment  and  outcomes 

• Integrate  new  science  knowledge  for  later  transfer  by  requiring  teachers  to  develop  a lesson  plan  during  the 
workshop 
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• Practice  the  lesson  plan,  collect  feedback,  and  revise  it  for  improvement 

• Use  formative  evaluation  at  3 levels  to  achieve  a more  complete  analysis:  program,  peer,  and  student. 

To  further  reinforce  transfer  of  knowledge  after  the  workshop,  the  teachers  were  required  to  attend  six 
follow-up  seminars  over  the  following  academic  year.  These  seminars  were  designed  to  provide  pedagogical  support 
for  delivering  the  lesson  in  the  teacher’s  classroom,  teach  emerging  technology  skills,  and  focus  more  in  depth  on 
STEM  careers. 


Results,  Recommendations  and  Future  Directions 

Preliminary  results  from  the  workshop  show  the  majority  of  teachers  were  very  satisfied  with  the 
experience  of  doing  science  the  way  real  scientists  do.  They  liked  collecting  mussels  using  carefully  controlled 
research  procedures  and  processing  the  specimens  under  the  watchful  eyes  of  the  marine  biologist  and  her  teaching 
assistants.  They  expressed  satisfaction  with  the  rigor  of  learning  how  to  carefully  handle  wet  lab  equipment, 
chemicals,  and  specimen  dissection  tools. 

Recommendations 

• Continue  to  use  marine  biotechnology  as  the  basis  for  the  immersion  experience  allowing  participants  to 
“be  scientists  and  to  do  real  science” 

• Continue  incentives  (stipend  and  follow-up  seminars)  to  encourage  high-levels  of  commitment  and 
increased  focus  on  the  learning  process 

• Increase  instructional  support  for  learning  the  science  content 

• Increase  instructional  technology  support  for  developing  and  implementing  the  lesson  plans 

• Allow  more  workshop  time  to  write  lesson  plans 

• Establish  unstructured  preparation  time  for  lesson  plan  development,  such  as  library  visits,  reflection, 
collaboration 

• Control  workshop  implementation  to  minimize  scope  creep  of  one  topic  into  the  time  allotted  for  another 
topic 

Future  Directions 

Given  the  outdated  science  standards  at  the  national  and  state  level,  the  results  of  this  and  the  two  future 
workshops  could  form  the  basis  for  a plan  to  initiate  a major  rewrite  of  the  national  science  standards.  In  addition, 
the  lesson  plans  and  instructional  products  that  emerge  from  the  three  workshops  could  form  a solid  secondary 
education  curriculum  for  marine  biotechnology  and  bioinformatics  and  be  used  to  initiate  science  curriculum  reform 
to  integrate  biotechnology  and  bioinformatics. 
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Supporting  Instructors  in  the  Transition  between  Learning  Management 

Systems 
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Abstract 

One  of  the  most  challenging  technological  tasks  facing  college  faculty  today  is  transitioning  to  a new  Learning 
Management  System  (LMS)  or  from  one  version  of  an  LMS  to  another.  The  speed  at  which  the  development  of  a 
given  LMS  progresses  often  outpaces  the  ability  of  faculty  to  easily  adapt.  New  features  and  functionality  may 
appear,  and  elements  of  the  graphical  user  interface  may  change.  These  differences  between  LMSs,  or  versions  of 
LMSs,  can  be  costly  to  a faculty  member,  in  terms  of  time,  anxiety  and  effort.  Helping  faculty  to  make  this 
transition  is  by  definition  a change  management  process,  and  we  will  also  discuss  a useful  technique,  Force  Field 
Analysis,  which  can  help  instructional  support  staff  identify  and  evaluate  aspects  of  the  overall  situation  that  help  or 
hinder  the  successful  transition  by  faculty.  We  also  provide  a description  of  our  Multidimensional  Support  Model 
(MSM)  for  instructional  support  staff  to  utilize  as  they  help  faculty  survive  this  transition.  The  MSM  focuses  on 
implementation  concerns,  communication,  and  training  issues  within  a socio-technical  system  (STS)  such  as  an 
LMS.  We  describe  our  activities  as  primary  support  staff  during  a recent  ongoing  LMS  transition,  and  present  a 
model  for  supporting  faculty  as  they  transition  between  different  LMSs,  or  substantially  different  versions  of  the 
same  LMS.  Our  MSM  model  highlights  the  importance  of  a coordinated  effort  in  communications,  information 
sharing,  and  training  between  various  support  centers,  on  multiple  campuses. 

Context  of  Study 

Our  institution,  a large  Midwestern  state  university,  is  currently  in  the  final  stages  of  a transition  between 
different  LMSs,  the  second  such  transition  in  five  years.  While  the  systems  are  similarly  named,  Oncourse  and 
Oncourse  CL,  they  are  actually  completely  different,  both  in  user  interface  and  technological  underpinnings.  What 
makes  this  transition  particularly  challenging  is  that  our  new  LMS,  Oncourse  CL,  is  under  active  technical 
development  as  part  of  the  Sakai  Project  (http://www.sakaiproject.org),  an  open-source  software  development  effort 
that  builds  on  the  Open  Knowledge  Initiative  (OKI)  framework  (Fuchs,  2004),  and  involves  several  universities 
sharing  in  the  development  effort.  Although  this  provides  considerable  advantages  to  the  institution  in  sharing  of 
resources,  costs,  and  knowledge,  it  also  introduces  many  additional  challenges  for  faculty  and  support  staff.  Among 
these  challenges  is  the  on  going  development  process  itself,  which  results  in  faculty  often  encountering  new 
features,  functionality,  and  related  bugs. 


Introduction 

Learning  Management  Systems  (LMS),  sometimes  referred  to  as  Course  Management  Systems  (CMS),  are 
ubiquitous  on  today's  college  campuses.  Morgan  (2003)  defines  a CMS  as  "a  software  system  that  is  specifically 
designed  and  marketed  for  faculty  and  student  use  in  teaching  and  learning."  Watson  et  al.  (2007)  define  an  LMS  as 
"an  infrastructure  that  supports  the  delivery  and  management  of  instructional  content,  the  identification  and 
assessment  of  individual  and  organizational  learning  goals,  and  the  management  of  the  progression  toward  meeting 
those  goals".  A typical  LMS  is  quite  complex,  generally  incorporating  communication  tools  such  as  discussion 
forums,  email  systems  and  chat  rooms,  administrative  tools  such  as  drop  boxes,  grade  rosters  and  course 
documentation,  and  interactive  functions  such  as  online  testing  and  quizzing,  automatic  grading  and  e-portfolios. 

An  LMS  is  arguably  the  most  complex  software  system  that  college  faculty  face  in  their  day-to-day  professional 
lives.  Add  to  this  the  reality  that  many  systems  are  in  a constant  state  of  change,  due  to  various  socio-technical 
system  components  that  influences  their  design  and  development,  such  as  the  instructional  needs  of  faculty  and 
students  (Morgan,  2003).  An  LMS  may  also  be  revised  to  accommodate  improvements  in  technology,  or  technical 
and  administrative  priorities.  Because  of  this,  faculty  often  find  themselves  in  the  situation  of  having  mastered  one 
version  of  an  LMS  and  now  having  to  take  on  another.  Given  the  busy  life  of  the  typical  college  faculty  member, 
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with  teaching,  research,  service,  and  family  responsibilities,  they  often  find  it  difficult  to  make  time  in  their 
schedules  to  attend  lengthy  and  exhaustive  training  seminars  to  learn  all  the  complexities  of  a new  substantially 
revised  LMS.  Sometimes  faculty  are  hesitant  of  fully  utilizing  an  LMS,  in  part,  out  of  concern  that  it  will  change  or 
not  be  available  to  them.  The  complexities  of  an  LMS  transition,  the  overall  complexity  of  a socio-technical  system, 
and  issues  with  administrative  and  faculty  communication  are  all  significant  challenges  faced  by  instructional 
support  staff  in  the  training  and  support  process.  Morgan  (2003)  thus  cites  improvements  in  communication 
between  administrators  and  faculty,  the  importance  of  minimizing  disruptions,  and  the  creation  of  policies  and 
change  management  practices  as  critical  steps  in  supporting  and  encouraging  faculty  use  of  an  LMS. 

As  instructional  support  staff,  we  have  faced  the  challenge  of  bringing  faculty  across  the  gap  between  different 
LMS  versions  or  systems.  We  believe  that  other  instructional  support  professionals  will  find  our  MSM  model 
helpful  as  they  design  their  own  LMS  transition  strategy.  College  faculty  will  also  benefit  from  learning  about  our 
model  and  related  techniques,  so  that  they  can  be  informed  about  the  support  that  they  will  need  during  an  LMS 
transition,  and  will  be  prepared  to  request  it,  if  not  already  available,  from  their  institution. 

LMS  as  a Socio-Technical  System 

In  this  effort,  we  utilize  a socio-technical  approach  as  our  primary  theoretical  perspective.  This  approach 
suggests  that  the  social  and  technical  components  of  a system  are  closely  linked.  It  further  suggests  that  both  the 
social  and  technical  aspects  are  equally  important  when  implementing,  using,  or  evaluating  information  technology. 
Kling  supports  this  concept  of  a socio-technical  interaction  network  (STIN),  noting  that,  "the  STIN  model  shares  the 
views  of  many  socio-technical  theories:  that  technology-in-use  and  a social  world  are  not  separate  entities — they  co- 
constitute each  other”  (Kling,  et  al.,  2003).  The  abstraction  of  an  LMS  as  a socio-technical  system  is  further 
supported  by  Moore  & Kearsley  (1996),  in  which  they  explain  that  learning  management  systems  are  complex 
systems  that  involve  a wide  variety  of  technological,  organizational,  social,  and  instructional  components.  Taken 
alone,  the  design  and  implementation  of  an  LMS  is  not  a simple  task.  Combining  that  together  with  a transition 
from  one  LMS  to  another,  increases  the  overall  complexity  exponentially,  thus  increasing  the  cognitive  load  and 
anxiety  on  many  of  the  social  actors  within  the  STS;  including  instructional  support  professionals,  faculty,  students, 
and  administrators.  It  also  increases  the  importance  of  knowing  the  components  of  a system,  the  goals  for  the 
transition,  and  the  forces  that  for  and  against  the  change  effort.  In  an  effort  to  illustrate  a LMS  as  socio-technical 
system,  it  is  useful  to  note  the  important  social  and  technological  aspects  that  define  it.  An  STS  is  comprised  of 
identifiable  populations,  groups,  incentives,  actors,  undesired  interactions,  flows,  and  choice  points  (Kling,  et  al., 
2003).  It  is  all  of  these  components  that  encompass  a learning  management  system.  It  is  our  belief  that  having  a 
complete  socio-technical  representation  of  our  system  will  improve  decision-making,  and  the  resulting  policies, 
procedures,  and  support  model. 

In  devising  our  support  strategy  our  first  step  of  analysis  was  to  determine  the  components  and  influencing 
factors  of  the  LMS  transition.  After  viewing  our  LMS  through  the  prism  of  the  STS  approach,  it  became  easier  to 
identify  the  driving  and  restraining  forces  that  may  impact  the  transition.  Assessing  these  forces  prior  to  the  actual 
implementation  allows  for  the  prioritization  of  limited  support  and  training  resources. 

Force  Field  Analysis 

A Force  Field  Analysis  (FFA)  was  performed  to  identify  the  major  forces  for  and  against  the  change  effort. 

Force  Field  Analysis,  developed  by  Kurt  Lewin  (1943),  is  a common  diagnostic  tool  used  in  many  change 
management  efforts.  The  result  of  an  FFA  analysis,  if  positive  (the  driving  forces  outweighing  the  restraining 
forces)  suggests  that  the  change  effort  will  be  successful,  and  where  to  position  specific  energy  and  effort  in  order  to 
effect  positive  change  or  minimize  difficulties.  When  creating  an  FFA  diagram,  the  positive  and  negative  forces  are 
identified  and  then  given  relative  weights  of  importance.  Thomas  (1985)  notes  that  the  utilization  of  FFA  is  present 
in  numerous  contexts,  but  that  it  was  rarely  applied  to  strategy.  He  goes  on  to  state  that  FFA  could  provide  new 
insights  into  the  evaluation  and  implementation  of  corporate  strategies.  We  chose  to  use  an  FFA  analysis  (figure  1) 
to  help  focus  our  training  and  support  strategy  at  a local  level.  We  based  the  FFA  analysis  upon  our  daily 
interactions  as  instructional  support  personnel  with  faculty  who  were  using  our  LMS.  While  admittedly  a subjective 
measure,  by  using  FFA  we  were  able  to  identify  and  prioritize  training  and  other  support  efforts  more  effectively,  as 
well  as  prepare  for  potential  restraining  forces  in  an  effort  to  minimize  those  forces. 
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Figure  1-  Force  Field  Analyses 
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The  Multidimensional  Support  Model 

Our  MSM  model  (figure  2)  is  an  attempt  to  clarify  our  support  and  communication  techniques  and  brings 
together  many  of  the  practical  and  theoretical  underpinnings  of  our  overall  strategy.  The  model  reflects  a clear  and 
flexible  timeline,  the  importance  of  communication  between  the  support  centers,  the  teaching  and  training  centers, 
and  the  system  development  team,  as  well  as  the  importance  of  communicating  with  faculty  in  regard  to  support  and 
training.  The  model  also  illustrates  the  reciprocal  or  iterative  nature  of  the  process  by  indicating  bi-directional 
communication  between  all  institutional  support  teams,  and  the  primary  users;  in  this  case  the  faculty.  A key  aspect 
of  the  MSM  model  is  the  multidimensionality,  which  is  carried  through  the  entire  model  in  the  coordination  and 
communication  of  the  support  staff  and  development  team,  and  in  the  multidimensionality  of  modes  of  support  and 
training  for  faculty. 

Clear  and  Flexible  Timeline 

A clear  and  flexible  timeline  set  forth  by  the  development  team  and  administrators  is  another  important  element 
in  our  support  model.  The  clarity  of  the  timeline  would  speak  for  itself  in  terms  of  the  importance  of 
communicating  with  faculty  about  specific  dates,  and  when  changes  will  take  place.  Even  more  important  is  the 
willingness  of  technical  and  institutional  administrators  to  recognize  potential  problems  in  the  transition,  and  to  take 
action  by  shifting  or  extending  the  timeline  in  order  to  alleviate  any  potential  problems.  This  situation  occurred  at 
our  institution  where  the  administration  extended  the  transition  timeline  after  collecting  feedback  from  faculty  and 
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support  professionals.  This  willingness  to  shift  the  transition  timeline  to  continue  to  operate  the  current  and  new 
system  for  a longer  period  of  time  helped  ease  faculty  stress,  and  minimize  disruption. 

Figure  2-  Multidimensionality  Support  Model 
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Support  Coordination  & Communication 

The  key  aspect  of  multidimensionality  in  the  MSM  model  is  demonstrated  in  the  coordination  and  communication 
between  support  staff  and  the  LMS  development  team.  The  continual  flow  of  information  between  the  various 
support  centers  and  campuses  is  carried  out  through  various  face-to-face  and  technological  channels.  This 
communication  is  multidimensional,  iterative,  and  bi-directional.  It  continually  precedes  and  follows  major  and 
minor  system  updates,  training  sessions,  bug  reports,  and  other  feedback  received  from  faculty  and  students.  An 
array  of  media  resources  is  utilized  to  maintain  continual  communication  and  coordination  including  the  use  of 
phone,  instant  messaging,  email,  blogs,  wikis,  online  knowledge  bases,  print-format  job  aid  documents,  and  large 
and  small-scale  meetings  via  videoconferencing.  All  of  these  media  are  critical  elements  of  the  coordination  and 
communication  strategy.  A positive  side  effect  of  using  these  online  technologies  is  that  faculty  also  have  access  to 
many  of  these  same  support  resources,  thus  reducing  communication  lag  time,  and  increasing  transparency  in  the 
support  process. 

Faculty  Training  & Communication 

Due  to  the  complexity  of  our  situation,  the  training  component  of  our  support  model  was  influenced  by 
Reigeluth’s  Elaboration  Theory  (Reigeluth  & Stein,  1983).  This  theory  provides  guidance  in  designing  instruction 
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and  encourages  the  teaching  of  complex  ideas  and  procedures  by  first  presenting  the  learner  with  the  simplest  form 
of  the  task,  or  the  epitome,  and  then  later  presenting  more  complicated  tasks  or  versions  of  the  task.  In  any  LMS, 
there  are  often  numerous  ways  of  accomplishing  a certain  instructional  goal,  some  of  them  more  complex  than 
others.  In  our  training,  which  is  necessarily  procedural  in  structure,  we  chose  to  teach  first  the  most  basic,  simplest 
ways  of  operating  in  the  system,  and  then  later  provided  more  complex  directions  for  more  challenging  tasks  when  a 
user  is  confident  and  ready.  We  chose  this  model  for  its  ability  to  reduce  complexity  in  the  learning  task,  with  the 
hope  that  it  would  also  reduce  faculty  stress  and  anxiety  by  alleviating  some  of  the  complexity  of  the  overall  system 
transition. 

The  aspect  of  multidimensionality  with  our  model  is  demonstrated  in  the  support  that  precedes  and  follows  the 
group  training  sessions  and  one-on-one  tutorials.  We  utilized  an  array  of  media  resources  to  raise  awareness  of  the 
changes  in  the  LMS  and  then  to  start  faculty  along  the  path  to  adoption.  The  use  of  email  announcements,  print  job 
aid  documents,  animated  Flash  tutorials,  online  support  knowledge  bases,  virtual  office  hours  (offered  via 
Macromedia  Breeze),  live  synchronous  text-based  chat  and  traditional  phone  support,  as  well  as  related  blogs  and 
podcasts  were  all  critical  elements  of  the  communication  and  training  strategy.  This  array  of  media  resources,  along 
with  the  face-to-face,  individual  consultations  and  group  sessions,  allowed  faculty  a multiplicity  of  means  to  get 
support,  which  helped  them  in  getting  started,  minimizing  their  stress,  and  successfully  utilizing  the  new  and/or 
improved  LMS  with  their  students. 


Limitations  and  Further  Research 

This  paper  is  based  on  our  own  experiences  working  as  instructional  support  personnel  at  a large  state  institution 
in  the  Midwest.  It  is  possible  that  instructional  support  staff  at  other  types  of  institutions  may  have  different  results 
following  our  model,  and  may  have  reasons  to  adapt  it  to  their  own  contexts.  Smaller  institutions,  for  example,  may 
find  communication  with  faculty  to  be  easier,  and  to  require  fewer  delivery  methods.  If  the  transition  between  LMS 
versions  focuses  on  just  a few  features,  is  purely  technological,  or  is  just  cosmetic,  there  may  not  be  the  need  for 
such  a complicated  transition  strategy.  Other  factors,  such  as  faculty  use  patterns,  size  of  courses,  makeup  of  the 
student  body,  etc.,  may  impact  LMS  use  and  thus  the  transition  strategy. 

Further  research  in  this  area  is  obviously  needed.  A study  that  collects  the  experiences  of  support  staff  across 
numerous  institutions,  and  types  of  institutions,  would  help  in  the  effort  to  refine  this  model,  as  would  studies 
featuring  a variety  of  LMSs.  Surveying  faculty  directly,  or  interviewing  them,  might  also  help  to  better  assess  the 
factors  in  the  FFA  analysis. 


Conclusion 

This  article  offers  the  reader  a glimpse  into  the  on-the-ground  realities  of  an  LMS  transition  at  a large 
Midwestern  state  university.  As  online  instructional  environments  are  constantly  in  flux  due  to  efforts  to  incorporate 
innovative  technologies  and  tools,  and  to  function  within  an  ever-changing  technological  infrastructure,  a clear 
vision  related  to  supporting  faculty  through  these  changes  is  a necessity  for  support  staff.  We  present  our  MSM 
transition  model  with  the  hope  that  our  experience  developing  it  will  assist  and  inspire  instructional  support  staff  at 
other  universities  as  they  strive  to  help  faculty  members  reach  their  goals  as  they  face  similar  transitions. 
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Supporting  Instructors 
through  a personal  support  system 

Gerard  J.  Gervedink  Nijhuis, 
University  of  Twente 


Summary 

Instructors  are  using  E-leaming  but  face  new  pressures  for  instance  to  perform  more  efficiently  and 
offer  their  teaching  in  ways  they  are  unfamiliar  with.  In  a multi-method  research  project  the  activities  of 
university  instructors  related  to  E-leaming  were  analyzed.  An  international  survey  was  one  of  the  research 
methods.  A Personal  Performance  Model  of  instructors  was  developed.  Based  on  this  model  a Personal 
Perfonnance  Support  Tool  was  designed  and  developed.  In  four  rounds  of  formative  evaluation  the  usability 
was  investigated  and  improved.  This  research  shows  that  such  a performance  support  tool  can  offer  instructors 
more  control  and  support  on  their  work,  time  management  and  course  quality  in  this  new  era  of  E-teaching. 

Pressure  on  Instructors 

External  demands  from  society,  students,  and  government  force  universities  to  change  their  mission 
statements  and  to  define  new  targets  for  teaching  and  learning.  Currently  education  in  Europe  is  in  transition 
towards  a new  program  structure.  This  Bachelor/Master  structure  implies  that  program  structure  and  current 
courses  will  change  to  fit  within  new  demands.  It  will  mean  offering  new  programs  to  a much  more -diverse 
student  population  with  various  backgrounds  coming  from  various  countries  in  Europe.  This  change  in  student 
demographics  will  also  increase  the  need  for  external  flexibility  in  offering  education  services,  especially  online. 
At  the  same  time  there  is  much  more  emphasis  on  quality,  both  from  government  and  society,  to  offer  good- 
quality  education  within  time  and  budget.  Schools  and  universities  have  to  set  up  strict  administrative 
procedures  to  audit  finances  and  quality,  and  they  are  held  accountable  for  students’  results.  These  tight  budget 
constraints  also  lead  to  increased  accountability  for  faculty  (Fisser,  2001). 

Chevaillier  (2000)  observes  that  "all  the  changes  that  have  affected  higher  education  systems  in  the  last 
three  decades  around  the  world  have  inevitably  had  an  impact  on  the  working  and  living  conditions  of 
academics."  (p.  28).  As  an  overview,  some  of  the  changes  are  depicted  as  pressures  on  faculty  in  Figure  1. 


Bachelor/ 
Master 
Virnplementation 


More- 

flexible 

programs 


Figure  1.  Examples  of  changes  that  will  affect  performance  of  instructors  as  a group. 


The  use  of  Course  Management  Systems  (CMSs)  such  as  BlackBoard,  WebCT,  Learning  Space,  and 
TeleTOP , has  greatly  facilitated  distance  education  and  blended  learning  (Collis  & Moonen,  2001),  but 
instructors  have  to  leam  how  best  to  offer  their  courses  using  such  a system  and  become  efficient  at  managing 
their  time  and  teaching-related  activities  involving  the  system  (Gervedink  Nijhuis  & Collis,  2003).  In  general, 
academics  usually  do  not  welcome  such  substantial  changes  with  their  accompanying  risks  and  uncertainty  (De 
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Boer,  2003).  Academics  indicate  that  they  feel  stressed  caused  by  increasing  workloads  and  therefore  sometimes 
leave  the  profession  (Meyer,  1998)  or  do  not  want  to  invest  time  in  developing  new  forms  of  education  (Collis 
& Messing,  2001).  These  observations  lead  to  a multi-method  research  project  that  has  been  running  from  2001 
to  2005  to  investigate  how  instructors  can  be  supported  when  performing  teaching-related  activities  in  this 
climate  of  changing  expectations  (Gervedink  Nijhuis,  2005). 

As  these  change  processes  are  still  evolving,  pressure  on  academics  will  continue  to  increase. 
Simultaneously,  academics  are  being  held  more  and  more  responsible  and  accountable  for  their  own 
performance.  Yet,  their  workload  is  already  (too)  high  and  satisfaction  (too)  low  which  influences  job 
performance  and  teaching  quality  in  a negative  way.  How  will  instructors  cope  with  these  changes  and  pressures 
and  perform  well  as  a group  and  at  a personal  level?  This  lead  to  the  initial  problem  statement  for  the  research: 
How  can  an  individual  instructor  stay  in  control  of  these  pressures? 

Research  Approach 

The  Human  Performance  Technology  (HPT)  methodology  (Stolovitch  & Keeps,  1999)  was  used  as  a 
framework  to  analyze  the  instructor’s  teaching-related  activities  (Figure  2).  Within  the  teaching  task  of 
academics,  two  major  categories  of  activities  can  be  distinguished:  teaching  activities  that  are  directly  linked  to 
student  learning,  such  as  delivery  of  face-to-face  sessions;  and  teaching-related  activities  to  manage  and  control 
the  instructor’s  personal  performance,  such  as  planning  and  administration.  Using  a CMS  makes  these  teaching- 
related  activities  a substantial  part  of  instructors'  work. 


Figure  2.  Schematic  HPT  model,  based  on  Stolovitch  and  Keeps  (1999). 


The  steps  are  shown  as  iterative  processes  in  which  the  problem  is  analyzed  and  causes  are  identified, 
followed  by  a development  process  to  create  an  intervention  which  is  then  refined  until  an  optimal  state  is 
reached.  This  part  of  the  HPT  process  can  well  be  addressed  by  the  development-research  approach  (Van  den 
Akker,  1999;  Reeves,  2000),  in  which  four  major  steps  are  defined:  analysis  of  needs,  development  of  solutions, 
evaluation  and  testing,  and  documentation  and  reflection.  In  the  development-research  approach  (Reeves,  2000) 
the  practical  problems  are  analyzed  by  researchers  and  practitioners.  The  close  involvement  of  practitioners  in 
research  both  as  objects  of  study  and  as  researchers  is  also  the  focus  point  in  Action  Research,  sometimes 
known  as  participatory  research.  In  Action  Research  the  researcher  is  placed  in  a position  as  a partner  "working 
alongside  those  affected  by  the  problem,  rather  than  as  an  objective  observer  who  might  impose  changes  from 
the  outside"  (Ryder  & Wilson,  1997,  p.  2).  In  this  research  the  researcher  and  the  practitioner  were  united  in  one 
person  and  his  reflective  insights  formed  the  start  of  the  development  process. 

In  a multi-method  research  approach,  data  was  collected  through  interviews  (sometimes  also  using  card 
sorting)  and  questionnaires,  and  through  analyzing  timesheets  of  instructors,  log  files  of  CMS  usage  by 
instructors,  and  CMS  content  entered  by  instructors.  An  international  survey  was  carried  out  to  investigate 
faculty  needs  and  support  in  6 European  countries  and  Australia. 

As  an  example  from  the  international  survey,  one  of  the  findings  about  the  level  of  support  offered  to 
instructors  is  shown  in  Table  1. 
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Table  1.  Current  level  of  support  for  instructors  using  ICT  for  teaching  purposes. 


In  your  opinion,  the  level  of  support 
for  instructors  with  respect  to  the  use 
of  ICT  for  teaching  purposes  in  your 
institution  is? 

Instructors 

(n=347) 

Decision 

makers 

(77=189) 

Support 

staff 

(77=154) 

Total 

(7V=690) 

M(SD ) 

- current 

- predicted 

M(SD) 

- current 

M(SD) 

- current 

M(SD ) 

- current 

Level  of  support  for  instructors 

2.96  (1.09) 
3.37  (.98) 

3.14(1.01) 

3.15(1.06) 

3.03  (1.06) 

Note.  Scale:  l=very  low,  3=average,  5=very  high. 


Table  1 shows  that  the  current  level  of  support  available  for  instructors  with  respect  tot  the  use  of  ICT 
for  teaching  purposes  scores  about  average.  Instructors  are  significantly  less  positive  (/?<.05)  than  the  other  two 
groups  as  according  to  decision  makers  and  support  staff  the  current  level  is  slightly  above  average.  Instructors 
expect  that  in  the  near  future  this  support  will  significantly  improve  (/?<.05),  although  still  just  above  average 
(M=  3.37).  Overall  the  level  of  support  is  valued  as  average  which  indicates  that  more  support  should  be  offered 
to  instructors. 


Conceptual  Model 

A conceptual  model  of  instructor’s  performance  was  developed.  The  model  shows  the  teaching-related 
performance  of  an  individual  academic  in  the  context  of  his  or  her  job,  with  work  relations  in  four  'directions'. 


Figure  3.  Conceptual  model  of  Personal  Performance 

The  first  direction  (A)  is  a direct  link  between  the  personal  characteristics  of  the  individual  academic 
and  job  performance.  A second  direction  (U)  indicates  academic's  performance  within  a specific  university 
context.  A third  direction  (C)  indicates  collaboration  with  colleagues  and  support  staff,  and  a fourth  direction 
(T)  indicates  direct  teaching,  such  as  face-to-face  sessions,  combined  with  the  teaching  and  management  of 
online  and  blended  courses  within  a CMS  such  as  Blackboard.  Being  able  to  manage  and  control  their  own  work 
within  the  dynamics  of  this  model  is  an  important  way  that  academics  can  improve  the  efficiency  and 
effectiveness  of  their  personal  performance,  and  one  or  more  support  interventions  (I)  can  offer  support. 

While  the  conceptual  analysis  identified  desired  performance,  a series  of  studies  was  carried  out  to  map 
actual  performance,  and  identify  gaps  and  their  causes.  The  results  of  these  studies  gave  a basis  for  comparing 
the  desired  and  actual  personal  performance  of  academics  in  terms  of  their  teaching-related  activities  and 
identified  potentially  serious  gaps  in  terms  of  maintaining  control  over  time  and  task  management. 

Results  showed  that  instructors  are  willing  to  change  but  do  not  have  the  right  tools  and  support 
available.  Also  the  universities  are  not  offering  on-going  support  to  handle  managerial  and  organizational 
support  for  the  individual  instructor.  The  international  survey  showed  that  instructors  indicate  that  in  the  future, 
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increasing  efficiency  will  become  even  more  important  and  that  ICT  use  is  expected  to  be  even  more  important 
in  the  future  than  today,  both  for  work  and  teaching.  The  study  also  showed  that  through  efficient  and  effective 
management  of  a course  environment  (that  is  part  of  a CMS),  does  not  yet  receive  attention  in  staffing  policies 
and  that  the  necessary  incentives  and  rewards  for  staff  to  make  the  effort  to  handle  a CMS  in  a well-managed 
way  are  lacking  (Gervedink  Nijhuis,  2002).  Instructors  also  indicate  that  they  want  to  be  well  organized  and 
want  to  plan  their  time  in  advance.  This  could  imply  that  instructors  can  benefit  from  a planning  tool  that 
supports  them  to  plan  their  time  in  advance. 

Personal  Performance  Support  System 

An  intervention  was  designed,  using  design-research  methodology  (Reeves,  2000).  In  this  research  the 
intervention  took  the  form  of  a series  of  prototypes  of  a Web-based  support  tool.  These  prototypes  were 
designed,  developed,  and  evaluated  to  lead  to  a final,  elaborated  prototype  of  a Personal  Performance  Support 
tool  to  support  instructors  in  their  teaching-related  activities.  The  last  version  of  the  prototype  was  accompanied 
by  an  implementation  plan,  indicating  the  institutional  practices  needed  before  such  a tool  could  be  of  maximal 
value  for  instructors. 

Instructors  need  integrated  support  at  various  levels,  as  their  work  is  complex  and  when  performing 
instructors  use  various  resources,  knowledge,  and  heuristics.  As  Gery  and  Jezsik  (1999)  put  it:  "An  EPSS 
[Electronic  Performance  Support  System],  because  it  is  incorporated  into  the  work  process,  enables  individuals 
to  get  up  to  speed  quickly  and  work  more  efficiently.  It  manages  complexity  and  work  flow  for  the  user,  and  it  is 
capable  of  supporting  diverse  working  styles.  In  addition,  it  enables  performers  to  share  knowledge."  (p.  143). 
This  seems  quite  appropriate  for  instructors  to  improve  their  performance.  Therefore,  an  EPSS  which  may 
incorporate  aspects  of  expert  systems  and  job  aids  seems  the  most  beneficial. 

The  final  version  of  the  support  tool  incorporated  various  kinds  of  support,  such  as  background 
information,  checklists,  procedures  to  use  a CMS,  and  administrative  documents  and  forms.  This  support  is 
offered  in  a context-sensitive  manner  to  help  the  instructor  focus  on  the  relevant  parts.  The  user  interface  offers 
instructors  sequential  overviews  of  activities  to  perform,  but  specific  support  can  be  accessed  in  a direct  manner. 
An  example  of  a support  page  is  shown  in  Figure  4. 


3 Academics  in  Control  - Microsoft  Internet  Explorer 


E'le  Edit  View  Favorites  loots  Help 


HHHj 


Develop 

- Course  ... 

- Components  — 
’-  Lectures 

Assignments 
- Discussions 

Deliver 

- Course  ... 
Components 


Sitemap 
Research  ... 
Contact 


Academics  in  Control 

Personal  Performance  Support  Tool  (PPST)® 


3a.  Create  lectures  

I LacTuro  | 

Subtask  of:  Develop  course  components 


Subtask  of: 

Task  description: 


Create  a new  lecture  and.  put  it  in  TeleTOP.  and  add  before-lecture  and 
after-lecture  activities  Use  quiz  questions  during  a lecture  to  enhance 
interactivity 

The  common  activities  to  perform  this  task  are  shown  in  the  table  below 
Realistic  time  estimates  may  help  you  plan  your  time,  and  icons  link  to 


Activity  details 

Ref.  Req.  TeleTOP  Control 


specific  support 

Activities  overview  per  lecture 

* 

Activity 

Plan 

1 

Define  goals  and  objectives 

| 0.2  hrs 

2 

Select  topics 

| 0 2 hrs 

3 

Structure  lecture  and  class  activities 

| 0 5 hrs 

4 

Create  PowerPoint  presentation  and 
notes 

| 1 .5  hrs 

5 

Write  Before'  lecture  activities 

| 0 3 hrs 

6 

Write  'After'  lecture  activities 

| 0 3 hrs 

7 

Create  ourz  Questions 

| 0 hrs 

8 

Test  lecture 

| 01  hrs 

9 

Finish  lecture  development 

| 01  hrs 

Time  per  lecture 
Specify  number  of  lectures  in  the  course 

ruins 

■ 

Total 

| Todays 

Figure  4.  Example  of  a webpage  within  the  Personal  Performance  Support  Tool  offering  support  to  create  a 
lecture. 


327 


Research  Findings 


The  fonnative  evaluation  of  the  final  version  was  carried  out  by  instructors,  management  staff,  and 
support  staff  at  the  University  of  Twente.  The  potential  usefulness  as  rated  by  instructors  is  shown  in  Table  2. 


Table  2.  Overall  potential  usefulness  of  various  parts  of  the  PPST  by  instructors  («=1 1). 


Question 

Instructors 
M (SD) 

What  is  your  overall  impression  of  the  look  and  feel? 

3.91  (.54) 

What  is  your  overall  impression  of  the  screen  design? 

3.55  (.69) 

What  is  your  overall  impression  of  the  navigation? 

3.91  (.70) 

What  is  your  overall  impression  of  the  (potential)  usefulness  of  the  tasks 
and  activity  lists? 

4.36  (.50) 

To  what  extent  do  you  think  that  the  tasks  and  sub-tasks  match  common 
practice  of  instructors? 

3.45(1.21) 

What  is  your  overall  impression  of  the  content  of  the  PPST  tool  itself 
(excluding  the  content  of  resources,  such  as  articles,  documents,  forms)? 

4.18  (.60) 

What  is  your  overall  impression  of  the  (potential)  usefulness  of  the 
resources,  such  as  articles,  documents,  fonns,  that  are  linked  in  the  PPST 
tool? 

4.00  (.89) 

What  is  your  overall  impression  of  the  (potential)  usefulness  of  the 
planning  support? 

3.90  (1.20) 

What  is  your  overall  impression  of  the  overall  (potential)  usefulness  of  the 
support  tool  for  teaching-related  activities? 

4.00  (.89) 

Note.  l=very  negative;  3=neutral;  5=very  positive. 


Table  2 shows  Instructors  rated  the  potential  usefulness  of  various  aspects  of  the  support  tool  as 
positive  (M>3)  with  'tasks  and  activity  lists'  and  'content  of  the  tool'  as  more  than  positive,  and  'match  with 
common  practice'  as  more  than  neutral,  and  screen  design  also  rated  as  more  than  neutral.  The  PPST  overall 
impression  by  instructors  is  that  the  usefulness  of  the  tool  is  valued  as  highly  positive  (M=4.00,  SD=. 89).  This 
implies  that  the  PPST  has  the  potential  to  be  able  to  support  instructors  in  their  daily  work  and  make  it  more 
effective  and  efficient. 

Most  instructors  (8)  indicated  that  they  want  to  use  the  tool  or  at  least  give  it  a try  (2),  whereas  one 
instructor  would  only  want  to  use  some  "bits  and  pieces".  Two  educational  program  directors  indicated  that  they 
would  not  want  to  use  the  tool  but  in  contrast  one  director  indicated  a general  interest  in  using  new  tools. 
Support-staff  participants  were  very  willing  to  support  this  tool  and  submit  additional  references  and  guidelines 
to  the  tool. 

With  regard  to  the  control  issue  of  helping  academics  to  remain  in  control  of  their  time  and  teaching- 
related  planning  a majority  of  the  participants  (10)  felt  that  using  this  tool  would  result  in  instructors  being  more 
in  control  of  their  work.  As  one  instructor  indicated,  "it  will  offer  me  peace  of  mind",  and  another  instructor 
noted  that  using  this  tool  "may  prevent  some  planning  disasters  to  happen.".  One  support-staff  member  stressed 
the  ease  of  use  of  this  tool  as  a database  for  all  teaching-related  documents  and  other  related  material.  In 
contrast,  one  instructor  and  one  director  felt  that  this  tool  will  not  help  instructors,  as  this  tool  has  to  be  used  in 
addition  to  their  other  work,  and  they  are  already  overloaded. 

It  was  found  that  academics  also  value  in-person  support  when  it  is  available,  but  they  admitted  that 
availability  is  often  a problem  (not  only  during  the  evening  or  weekend,  but  also  during  working  hours).  Support 
staff  could  offer  a few  workshops  and  to  be  'at  hand'  when  needed,  although  others  just  want  to  start  using  the 
tool  and  find  out  themselves.  This  in-person  support  can  also  help  to  update  the  PPST  when  instructors  feel  the 
need,  or  organizational  procedures  change. 

After  a final  evaluation  of  the  Personal  Performance  Model  of  instructors  (Figure  3),  the  model  was 
amended  to  include  additional  aspects,  for  instance  the  fact  that  within  a university  daily  practice  within  various 
faculties  will  vary  and  a PPST  should  take  this  into  account  and  be  adaptable  to  meet  faculty-specific  needs.  To 
fill  the  PPST  with  useful  resources  various  'suppliers'  were  added,  such  as  directors  of  educational  programs, 
managers,  a librarian,  support  staff,  and  others  who  can  supply  documents,  procedures,  templates  and  more. 
Instructors  and  managers  also  suggested  adding  standard  procedures  to  indicate  that  within  workgroups 
collaboration  procedures  can  facilitate  easy  workflow.  Some  participants  also  suggested  that  directors  of 
educational  programs  should  be  involved  to  set  up  procedures.  Both  instructors  and  managers  emphasized  that 
this  Personal  Performance  Support  Tool  should  be  able  to  link  to  other  information  systems  at  a university  level, 
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such  as  Blackboard  or  a central  administrative  database,  to  offer  easy  access  and  prevent  the  need  to  submit 
information  twice.  As  the  tool  has  its  specific  functionalities  it  was  suggested  not  to  integrate  it  in  another 
system. 


Conclusion  and  Recommendations 

Instructors  and  managers  rated  the  potential  usefulness  of  the  PPST  as  high.  Instructors  are  very  willing 
to  give  it  a serious  try  and  indicate  that  using  this  tool  will  offer  them  more  control  for  their  teaching-related 
activities.  This  prototype  still  needs  additional  educational  components  and  a way  of  interacting  with  other 
systems  at  a university  level.  As  instructors  perfonn  their  work  within  a faculty-specific  environment  the  PPST 
should  also  be  adaptable  to  meet  the  daily  practice  of  the  faculty  involved.  This  means  that  the  PPST  should  be 
filled  with  faculty-specific  information,  procedures,  and  documents. 

Most  participants,  both  instructors  and  managers,  are  reluctant  about  actual  use  as  it  will  change  work 
procedures  and  current  practice  of  instructors.  They  emphasize  that  the  deployment  should  be  done  very 
carefully  and  gradually,  with  a PR  campaign  and  some  short  workshops.  They  also  indicate  that  personal 
support  is  required,  with  also  a procedure  to  suggest  amendments. 

A university  can  develop  incentives  for  the  use  of  this  tool  to  increase  the  individual’s  personal  control 
over  teaching-related  work  processes  and  to  improve  the  execution  of  administrative  processes.  Instructors  will 
demand  such  a tool  when  pressure  on  them  gets  higher  and  educational  demands  lead  to  an  overload  of 
expectations.  As  both  of  these  forces  are  building  on  the  individual  instructor,  this  will  increase  the  need  for 
such  a tool.  Universities  should  start  now  to  consider  the  implementation  of  such  a personal  performance 
support  tool.  In  this  way  instructors  can  stay  in  control  of  their  teaching-related  performance.  In  line  with 
design-research  methodology,  the  research  concludes  with  a series  of  guidelines  and  recommendations  that  can 
help  this  process  of  staying  in  control  to  occur.  In  this  research  17  recommendations  were  defined  on  how  to 
support  instructors  for  three  distinct  levels  within  a university:  the  organizational-management  level,  work- 
group management  level,  and  the  individual  instructor  level.  The  list  was  enhanced  with  recommendations  for 
researchers  of  human  performance  and  for  designers  of  support  systems  for  instructors.  Some  examples  of 
recommendations  are  listed  here. 

At  the  organizational-management  level  one  of  the  recommendations  is: 

Management  should  provide  instructors  with  clearly  stated  procedures,  guidelines,  and  templates 
to  streamline  workflow  and  administrative  procedures  within  faculties  and  workgroups. 

At  the  workgroup  management  level  one  of  the  recommendations  is: 

Workgroup  management  and  peers  should  stimulate  the  common  use  of  a PPST  to  streamline 
common  work  procedures  within  a workgroup. 

At  the  instructors’  level  two  of  the  recommendations  are: 

Instructors  should  work  in  a systematic  way,  using  a project-management  work  approach  and 
define  own  personal-work  procedures  in  order  to  control  effective  and  efficient  course  preparation 
and  delivery,  taking  into  account  their  work-group  and  university  constraints. 

Instructors  should  plan  their  time  in  a systematic  way  not  only  for  face-to-face  sessions  but  more- 
and-more  for  teaching-related  activities,  for  instance  planning  for  time  to  submit  feedback  to 
assignments  or  time  to  participate  in  online  discussions. 

The  research  suggests  that  when  the  list  of  recommendations  is  followed  through  it  will  create  a 
working  context  for  instructors  in  which  they  can  more  efficiently  perfonn  their  teaching-related  activities  and 
that  a PPST-type  tool  will  be  available  to  support  them  to  optimize  their  organization  and  management  of  their 
own  work.  Finally,  this  research  emphasizes  the  importance  of  offering  support  to  individuals  who  have  to 
perform  in  an  ever-changing  environment.  As  instructors  become  more-and-more  self-responsible  for  their  work 
and  have  to  perfonn  in  increasingly  efficient  and  effective  ways  and  are  being  held  accountable  for  good  quality 
courses  and  results,  they  will  need  to  be  able  to  get  support  to  stay  in  control. 
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Abstract 

In  this  collaborative  research  project,  six  middle  grades  science  teachers  have  co-deve loped  concept  mapping 
activities  tied  to  soil  and  air  quality  environmental  issues  in  the  state  curriculum.  Project  highlights  include  a two- 
day  professional  development  seminar  for  teachers,  collaborative  lesson  planning,  and  classroom  instruction 
designed  to  promote  student  thinking  skills  about  priority  environmental  issues.  The  project  is  supported  by  the 
freeware  Cmap  software  program  for  preparing  and  editing  Web-based  concept  maps.  A mixed  methods  approach  is 
employed  with  qualitative  cases  and  quantitative  control  groups.  Findings  will  demonstrate  the  type  and  quality  of 
student  thinking  elicited  by  mapping  activities,  as  well  as  potential  benefits  of  mapping  on  learning  outcomes. 

Purpose 

This  collaborative  research  project  between  a university  researcher  and  six  middle  grades  science  teachers 
promotes  students'  scientific  thinking  skills  through  Web-based  concept  mapping.  The  goal  of  the  project  is  to 
increase  student  awareness  of  selected  environmental  problems  in  the  community,  as  well  as  actions  that  can  be 
taken  to  alleviate  the  problems.  Teachers  select  topics  that  tie  into  the  state  curriculum,  with  soil  quality  and 
stewardship  taught  in  grade  6,  and  air  quality  and  stewardship  taught  in  grade  7.  Students  use  Web-based  concept 
mapping  software  to  organize  and  relate  term  sets. 

Our  state's  science  testing  at  the  end  of  the  8th  grade  has  an  increased  focus  on  processing  information  and 
higher-order  thinking  skills.  The  focus  of  this  project  aligns  with  these  emerging  requirements,  since  it  tasks  early 
middle  grades  students  with  processing  online  information  sources  through  the  basic  thinking  skills  of  qualifying, 
classifying,  and  finding  relationships  (Presseisen,  2001).  Building  this  foundation  of  thinking  skills  is  important  for 
future  science  classes  that  apply  end-of-grade  tests  based  on  thinking.  Concept  maps  are  one  of  several  techniques 
recommended  by  National  Science  Education  Standards  to  promote  student  thinking  (NRC,  1996,  p.  144):  "Students 
need  opportunities  to  present  their  abilities  and  understanding  and  to  use  the  knowledge  and  language  of  science  to 
communicate  scientific  explanations  and  ideas.  Writing,  labeling  drawings,  completing  concept  maps...  should  be  a 
part  of  science  education."  Mapping  tools  in  particular  support  recommended  learning  strategies  in  allowing 
students  to  build  on  what  they  already  know  about  a subject  and  revise  that  understanding  through  further  reading 
and  discussion  (Bransford,  Brown,  & Cocking,  1999). 

The  project  is  supported  through  Cmap  software  for  online  concept  mapping  (IHMC,  2006).  With  Cmap, 
teachers  can  establish  and  populate  student  Web  folders  with  raw  maps  for  editing  and  relevant  resources  to  assist 
with  the  task  (e.g.,  articles,  Web  links,  images).  Students  can  access  their  server-based  folders  from  any  Internet- 
connected  computer  that  has  the  freeware  Cmap  tools  software  program.  Students  can  create  new  concepts  or  work 
with  seeded  concept  sets  provided  by  the  instructor.  When  connecting  two  or  more  concepts  in  a map,  Cmap 
prompts  the  student  to  enter  a descriptive  relationship  phrase  or  "proposition"  statement,  consistent  with  the 
recommended  concept  mapping  approach  (Novak,  1998). 

Specific  project  objectives  include: 

Objective  1 : The  student  will  apply  basic  thinking  skills  to  deconstruct  environmental  problems:  a)  qualifying  or 
recognizing  basic  units  of  identity  (creating  concept  categories),  b)  classifying  or  sorting  pre-selected  resources  into 
concept  categories,  and  c)  finding  relationships  or  patterns  and  sequences  in  the  data. 

Objective  2:  Using  their  completed  concept  map,  the  student  will  engage  in  the  complex  thinking  process  of 
decision  making  to  make  recommendations  for  addressing  stated  environmental  problems. 
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Theoretical  Framework 


A void  exists  in  research  that  investigates  concept  mapping  as  an  educational  strategy  to  promote  student 
thinking  skills.  A review  of  archived  articles  related  to  "concept  mapping"  in  the  ERIC  Education  database  between 
2004-2005  reveals  literature  more  frequently  related  to: 

• non-research  descriptions  of  concept  mapping  as  a methodological  or  educational  tool  (Carnine  & Carnine, 

2004;  Goodyear,  Tracy,  Claiborn,  Lichtenberg,  & Wampold,  2005;  Lee,  2004;  Royer  & Royer,  2004) 

• applications  of  mapping  to  address  specific  needs  such  as  sorting  focus  group  findings  (Cousineau,  Goldstein,  & 
Franko,  2004),  sorting  information  to  help  pre-service  teachers  prepare  lessons  (Machin,  Valeys,  & Loxley, 
2004),  collapsing  multiple  maps  to  derive  a program  theory  that  depicts  group  thinking  (Rosas,  2005; 
Yampolskaya,  Nesman,  Hernandez,  & Koch,  2004),  deriving  remedial  instruction  paths  (Jong,  Lin,  Wu,  & 

Chan,  2004),  or  structuring  information  associated  with  writing  tasks  (Riley  & Ahlberg,  2004) 

• discussions  of  how  to  best  interpret  or  assess  concept  maps  (Jong  et  ah,  2004;  Passmore,  2004;  Riley  & 

Ahlberg,  2004;  Van  Zele,  Lenaerts,  & Wieme,  2004;  Yin,  Vanides,  Ruiz-Primo,  Ayala,  & Shavelson,  2004). 

Many  researchers  have  used  concept  maps  to  document  student  knowledge  structures,  but  with  different  topics, 
age  groups,  or  under  different  conditions  than  the  proposed  study.  Blake  (2004)  and  Lewin  (2004)  documented 
individual  understanding  of  rocks  and  home  technology  respectively.  Kinchin,  De-Leij,  and  Hay  (2005)  investigated 
the  use  of  maps  to  improve  undergraduate  integration  of  microbiology  course  material.  And  other  researchers  have 
modified  mapping  conditions  to  determine  if  different  source  texts  (Graff,  2005),  given  map  structures  (Lee  & 
Nelson,  2005),  or  alternative  control  structures  (Chiu,  2004)  impact  map  development.  Only  one  study  investigated 
similar  thinking  skills  to  the  proposed  study— MacGregor  and  Lou  (2004-2005)  discovered  concept  map  templates 
can  assist  with  recalling  and  applying  knowledge  from  Web  sources  during  inquiry  tasks. 

To  use  concept  maps  for  assessment,  at  least  three  things  must  be  accounted  for:  a task  with  directions,  some 
format  for  the  student's  response,  and  a scoring  system  (Ruiz-Primo  & Shavelson,  1996).  Tasks  vary  by  constraints, 
with  some  providing  pre-selected  terms  or  propositions  to  students  for  mapping,  and  others  asking  students  to 
generate  their  own  list  of  terms  or  propositions.  Response  formats  vary  by  mode,  including  paper  and  pencil, 
computer-generated,  and  interview.  Response  formats  also  vary  by  characteristics  and  difficulty,  with  some 
providing  an  easier  fill-in-the-blank  map  structure  for  students  on  a partially-created  map,  others  providing  computer 
pull-down  menus  with  relationship  statements  already  defined,  and  some  providing  no  map  elements  ahead  of  time. 

At  the  training  seminar,  teachers  were  shown  a continuum  of  mapping  activities  from  fill-in-the-blank  templates 
to  partially  completed  seeded  maps  to  completely  open-ended  maps,  with  the  recommendation  that  novice  students 
start  out  with  seeded  maps  until  they  understand  the  mapping  process  and  Cmap  tool.  Seeded  maps  also  provide  a 
common  term  set  to  score,  making  assessment  more  objective. 

More  than  20  different  scoring  strategies  are  available  for  maps  when  considering  the  variations  in  map  type  and 
the  multiple  map  items  that  can  be  scored  (i.e.,  concepts,  sets/hierarchies,  relationships,  cross-links,  resources) 
(Ruiz-Primo  & Shavelson,  1996).  A general  recommendation  is  to  focus  on  the  adequacy  of  propositions  as  the  most 
important  map  element,  while  downplaying  scoring  systems  that  simply  count  concepts  and  links  (Rye  & Rubba, 
2002).  In  this  study,  teachers  were  trained  to  score  student  maps  using  a combination  method  of  comparing  the 
student  map  to  their  own  instructor  map  (one  point  for  every  concept  grouped  similarly  to  the  instructor's  concept 
sets),  and  scoring  the  presence  of  selected  map  components  (one  point  for  each  correct  relationship  statement  and 
cross-link  statement  between  concept  sets). 


Method 

A faculty  research  grant  was  secured  from  the  university  during  spring  2006  to  support  teacher  stipends  and 
travel  to  an  on-campus  seminar  and  researcher  travel  to  school  sites.  Applications  were  solicited  from  6th  and  7th 
grade  science  teachers  across  the  state,  through  contacts  with  state  technology  and  science  education  teacher 
organizations.  Six  teachers  were  selected  from  different  schools  based  on  their  interest  in  graphic  organization  tools 
and  grouped  into  grade-level  teams  of  three  persons  each.  Cmap  server  software  was  installed  at  the  university, 
allowing  teachers  and  their  students  to  store  and  edit  their  concept  maps  and  attached  resources  entirely  online. 
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A 2-day  seminar  was  held  on  June  12-13,  2006,  to  train  the  cohort.  On  day  1,  the  cohort  received  direct  training 
on  Cmap  features  and  practice  generating  and  scoring  maps  with  sample  lessons.  On  day  2,  teachers  worked  in 
grade-level  teams  to  select  resources  and  develop  expert/instructor  maps  for  two  environmental  lessons.  The  6th 
grade  team  developed  two  instructor  maps  tied  to  soil  erosion,  while  the  7th  grade  team  developed  two  instructor 
maps  tied  to  air  pollution.  A student  version  of  each  map  was  then  generated  consisting  of  a few  seeded  terms  from 
the  instructor  map  and  the  remaining  pre-selected  terms  in  an  unstructured  list  for  the  students  to  organize  and  relate. 

Currently,  students  are  being  assigned  folders  and  passwords  on  the  Cmap  server  using  rosters  provided  by  each 
teacher.  Map  templates  and  applicable  resources  selected  by  the  teachers  during  planning  will  be  duplicated  and 
placed  in  student  folders.  The  free  Cmap  desktop  application  will  be  installed  on  each  internet-connected  computer 
from  which  students  will  be  constructing  their  maps.  Teachers  were  required  to  secure  the  signature  of  their  school 
technical  support  person,  agreeing  to  assist  with  these  installs,  as  part  of  their  application  for  the  seminar  and  study. 
Prior  to  their  first  lesson,  each  teacher  will  engage  students  in  a short  training  lesson  to  learn  both  Cmap  software 
features  and  general  mapping  procedures.  The  training  lesson  will  task  students  with  developing  appropriately 
structured  concept  maps  using  a simple  topic  (e.g.,  basketball  rules). 

This  is  a mixed  methods  study  with  a quasi-experimental  two-group  control  group  design  and  a qualitative  case 
study  design.  The  unit  of  analysis  is  a case  site  comprised  of  one  teacher  and  at  least  four  classes,  allowing  for  both 
quantitative  and  qualitative  comparisons  across  sites.  Each  teacher  will  assign  at  least  two  classes  to  a treatment 
group  and  two  classes  to  a control  group,  with  an  estimate  of  at  least  150  students  in  each  group  at  each  grade  level 
across  the  multiple  school  sites.  Both  groups  will  receive  a pre-test  before  each  of  two  lessons  to  measure  their  prior 
knowledge  of  relationships  in  the  instructor  maps  developed  by  teacher  teams.  Both  groups  will  receive  the  same 
instruction  in  the  form  of  readings  and  regular  lab  activities.  The  experimental  group  will  spend  up  to  two  extra  days 
per  lesson  working  with  Cmap,  organizing  and  relating  pre-selected  concepts.  Both  groups  will  complete  a decision- 
making worksheet  based  on  consequences  and  values,  asking  them  to  use  evidence  from  class  activities  to  justify 
relevant  solutions  (Swartz,  2001).  This  activity  will  be  followed  by  a class  discussion  on  the  best  solution  options. 
Finally,  both  groups  will  complete  a post-test  with  appended  survey  questions.  Additional  data  sources  include 
classroom  observations  at  each  school  site,  a teacher  journal  for  reflections,  and  a semi-structured  teacher  interview. 

Appropriate  statistical  techniques  such  as  analysis  of  variance  will  be  utilized  to  determine  any  differences  in 
mean  scores  between  pre  and  post  tests.  Descriptive  statistics  will  be  calculated  for  such  data  sources  as  student 
surveys  and  the  mean  number  of  correctly  identified  propositions  per  lesson.  NVivo  software  will  be  used  to  import, 
code,  and  sort  qualitative  data  sources  (i.e.,  student  survey  comments,  teacher  interview  data,  journals),  to  seek  any 
evidence  of  patterns  within  or  across  case  sites.  This  qualitative  data  should  be  helpful  to  interpret  the  findings 
regarding  any  pre-test  to  post-test  gain. 


Preliminary  Findings 

Findings  from  the  completed  training  include  reluctance  by  teachers  to  trust  their  students  with  mapping  of  a 
more  open-ended  nature.  Some  teachers  expressed  concern  that  their  students  would  not  understand  how  to  organize 
and  relate  concepts,  and  their  initial  student  maps  were  very  basic  with  only  4-6  terms  to  fill-in  on  a nearly 
completed  map  template.  Through  negotiation,  teachers  eventually  developed  seeded  maps  with  only  a few  starter 
terms  and  multiple  unsorted  terms  for  students  to  organize  and  relate.  The  potential  to  elicit  and  depict  student 
thinking  is  much  greater  in  seeded  maps  than  in  fill-in-the-blank  maps,  which  is  a conception  that  took  teachers 
some  time  and  practice  to  understand. 

One  of  the  instructor  maps  created  by  6th  grade  teachers  at  the  summer  seminar  is  depicted  in  Figure  1.  The 
subject  of  this  map  is  "topsoil  erosion,"  including  human  causes,  effects,  and  potential  solution  options.  While 
Figure  1 depicts  the  completed  instructor  map,  Figure  2 illustrates  the  seeded  map  provided  to  students.  A few  of  the 
overarching  header  terms  are  provided  on  the  seeded  map,  but  the  student  is  responsible  for  arranging  the  unsorted 
terms  using  "like"  groupings  and  arrows  in  the  space.  The  student  must  then  generate  written  proposition  statements 
to  explain  the  relationship  between  each  set  of  connected  terms.  These  activities  are  tied  to  specific  thinking  skills — 
classifying  and  finding  relationships  (Presseisen,  2001).  As  noted  previously,  students  will  receive  one  point  for 
grouping  concepts  similarly  to  the  instructor  (e.g.,  increased  food  production  grouped  with  farming  practices  = 1 
point,  solid  waste  and  nuclear  waste  grouped  with  landfills  = 2 points).  Students  also  receive  one  point  for  each 
correctly  stated  proposition/relationship  statement,  and  one  point  for  cross-links  that  go  beyond  the  most  basic 
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relationships.  Pre-  and  post-tests  were  developed  on  the  basis  of  relationships  in  the  instructor  map.  Based  on  the 
map  in  Figure  1,  sample  test  items  include: 

1 . Due  to  growing  population,  loss  of  topsoil  is  caused  by: 

a.  animal  waste  b.  increased  food  production  c.  deposition  of  materials  d.  use  good  farming  tools 

2.  Which  of  the  following  factors  is  directly  related  to  construction  and  industry: 
a.  landfills  b.  glaciers  c.  slope  d.  precipitation 


Figure  1.  Sixth  grade  instructor  map  on  human  sources  of  topsoil  erosion. 
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Figure  2.  Sixth  grade  seeded  map  on  human  sources  of  topsoil  erosion. 

Figures  3 and  4 illustrate  one  of  the  instructor  maps  and  student  seeded  maps  created  by  7th  grade  teachers  at 
the  summer  seminar.  The  subject  is  human  sources  of  air  pollution.  As  with  the  6th  grade  map  discussed  previously, 
the  maps  emphasize  causes  of  the  air  pollution  problem,  negative  effects,  and  solution  options.  Pre-  and  post-test 
items  were  once  again  based  on  relationships  in  the  map.  For  example: 

1.  Ozone  in  the  lower  atmosphere  causes  all  of  the  following  problems,  except: 

a.  lower  crop  yield  b.  disruption  of  water  cycle  c.  death  of  fish  d.  severe  health  problems 


Figure  3.  Seventh  grade  instructor  map  on  human  sources  of  air  pollution. 

The  following  research  questions  will  be  addressed  by  the  instructional  portion  of  this  study,  commencing  Fall 
2006.  Multiple  data  sources  are  used  to  inform  each  question  as  shown  in  parentheses. 

1 . Flow  effectively  do  students  using  Cmap  classify  or  sort  and  categorize  pre-selected  resources  into  concept  units 
(Cmaps,  discussions,  observation,  journals)? 

2.  Flow  effectively  do  students  using  Cmap  find  relationships  or  the  patterns  and  sequences  outlined  in  pre- 
selected resources  (Cmaps,  discussions,  observation,  journals)? 

3.  Flow  appropriate  is  student  decision  making  in  terms  of  recommendations  made  to  solve  presented  problems 
(worksheets,  discussions,  observation,  journals)? 
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4.  How  much  do  students  learn  about  presented  problems  (pre-test,  post-test,  Cmaps,  survey  questions,  journals, 
interviews)? 

5.  How  effectively  can  teachers  prepare,  implement,  and  score  Web-based  concept  maps  in  their  instruction 
(observation,  journals,  interviews)? 


Figure  4.  Seventh  grade  student  map  on  human  sources  of  air  pollution. 

Importance 

Data  from  this  study  will  be  useful  to  detail  the  extent  and  quality  of  student  thinking  that  can  result  from 
mapping  activities  with  tool-based  scaffolds.  The  control  group  design  will  help  determine  if  mapping  activities  help 
students  learn  more  about  problem  relationships  than  students  who  don't  visually  organize  and  relate  terms. 
Qualitative  data  will  help  to  define  specific  enablers  or  barriers  to  these  learning  activities  across  six  different  school 
sites  and  instructors. 
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The  Evolution  of  the  Process  of  Online  Collaborative  Problem-Based 

Learning 
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Abstract 

The  purpose  of  this  study  was  to  examine  the  evolution  of  online  collaborative  problem-based 
learning  (PBL).  It  studied  two  semesters  of  PBL  implementation  in  a course  in  computer-mediated 
communication  (CMC).  Data  was  collected  from  students’  surveys,  course  instructor  interviews 
and  observations  of  the  learning  process.  The  results  highlighted  issues  related  to  the  scheduling  of 
activities,  supporting  technology  tools,  facilitation,  and  strategies  students  employed  during  the 
process  as  important  considerations  for  educators  when  implementing  online  collaborative  PBL. 

Introduction 

In  order  to  advance  the  understanding  of  online  collaborative  PBL,  this  investigation  looked  at  the  process 
in  a holistic  way  to  identify  the  areas  in  which  potential  adaptations  of  traditional  PBL  models  might  prove 
necessary.  The  research  had  the  purpose  of  answering  questions  related  to  the  development  of  the  overall  process  of 
collaborative  PBL  in  the  online  milieu  together  with  the  understanding  of  the  development  of  its  stages  (i.e.  group 
formation,  problem  understanding  and  refinement,  learning  and  application,  and  finalizing  the  solution  and 
presentation). 

The  study  began  with  the  analysis  of  an  online  course  for  its  potential  fitness  for  transformation  into  an 
online  collaborative  PBL  experience  according  to  characteristics  of  suitable  environments  for  PBL  (Nelson,  1999). 
The  analysis  yielded  the  course  suitable  in  terms  of  all  four  characteristics.  As  a result,  the  researcher  and  the  course 
facilitator  engaged  in  a collaborative  redesign  process  taking  into  consideration  the  literature  of  PBL.  Thus,  they 
adopted  an  ill-structured  design  problem  so  that  students  had  the  opportunity  to  fully  customize  it  to  their  interests.  It 
required  students  to  design  an  ensemble  of  computer-mediated  communication  means  to  support  the  communication 
needs  of  some  group  of  people  endeavoring  in  an  online  learning,  work  or  social  context.  The  problem  prompts  were 
redesigned  between  semesters  because  of  the  observations  from  Semester  1 . The  redesign  focused  on  modifying  the 
problems  to  communicate  more  clearly  the  idea  that  the  problems  were  about  designing  CMC  and  not  about 
designing  learning  experiences. 

In  order  to  form  groups,  course  designers  determined  that  students  would  group  themselves  by  participating 
in  a series  of  activities  that  would  enable  them  to  get  to  know  their  peers  and  their  interests  and  to  identify  common 
ground  for  collaborative  work.  Course  facilitators  also  modified  the  prompts  for  conversation  conducive  to  group 
formation  so  that  students  focused  on  the  type  of  context  in  which  they  were  interested  in  an  effort  to  avoid  focusing 
on  potential  solutions.  While  PBL  methodology  suggests  that  students  determine  the  learning  issues  and  find 
learning  resources  by  themselves,  the  literature  on  online  PBL  suggested  this  was  a problematic  aspect  (Taplin, 
2000).  Thus,  course  facilitators  provided  a core  of  resources  and  learning  issues  but  encouraged  students  to  acquire 
in-depth  understanding  of  the  particular  issues  that  were  relevant  to  their  problems.  There  was  no  direct  instruction 
but  an  exchange  of  ideas  in  the  group  as  a whole  with  regards  to  the  ideas  exposed  on  those  learning  resources 
throughout  the  duration  of  each  semester.  Small-group  members  then  incorporated  relevant  ideas  into  the  resolution 
of  their  particular  problems.  Additional  external  tools  complemented  those  available  in  the  course  management 
system  (CMS)  to  support  communication,  collaboration,  co-authoring,  distribution,  management,  and  reflection. 
Observations  from  Semester  1 made  design  modifications  necessary  in  order  to  incorporate  a tool  that  would  help 
hold  the  cumulative  knowledge  of  the  small  group.  Course  facilitators  also  introduced  online  calendars  as  a tool  to 
help  students  coordinate  synchronous  chats. 

Students  underwent  the  PBL  process  in  three  stages.  During  the  first  stage,  students  engaged  in  the 
refinement  of  the  problem  they  wanted  to  address  according  to  their  contexts  of  interest.  This  refinement  consisted 
of  specifying  the  audience  for  which  they  wanted  to  design  CMC,  the  communication  needs  of  such  audience,  the 
technology  infrastructure  they  had,  and  a rationale  for  using  CMC  to  support  those  communication  needs.  Students 
submitted  a proposal  at  the  end  of  this  stage.  This  constituted  a milestone  in  their  work.  Course  facilitators  provided 
feedback  to  these  proposals  which  mainly  consisted  of  requesting  the  clarification  of  ideas  and  considering  the  scope 
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of  the  projects.  Students  underwent  a second  stage  by  means  of  which  they  learned  and  applied  their  learning  to  the 
resolution  of  their  problems.  Thus,  they  used  communication  theory  to  describe  communication  in  their  specific 
contexts  and  to  justify  the  selection  of  tools  to  satisfy  those  needs.  Furthermore,  they  described  the  implementation 
of  tools  as  part  of  their  designs.  Before  the  end  of  this  stage,  students  submitted  a draft  of  their  ideas.  Course 
facilitators  provided  feedback  once  again.  During  the  stage  that  brought  closure  to  the  learning  efforts,  students 
finalized  their  solutions  and  prepared  presentations  for  their  peers  in  other  groups.  Classmates  reviewed  other 
projects  and  shared  comments  related  to  the  description  of  the  context,  audience,  and  communication  needs,  the  use 
of  theoiy  to  describe  communication  needs  and  to  justify  the  selection  of  tools,  and  the  proposed  implementation  of 
those  tools  to  satisfy  the  communication  needs  within  the  context.  Group  members  assessed  peers  in  areas  such  as 
knowledge  of  theory  and  tools,  and  their  contributions  to  group  work.  Finally,  students  reflected  on  the  experience 
once  during  the  process  and  once  at  the  end. 

The  study  looked  at  implementations  of  this  design  during  two  consecutive  semesters.  The  group  in 
Semester  1 consisted  of  30  students,  25  of  which  agreed  to  participate  in  the  study.  The  group  in  Semester  2 
consisted  of  15  students;  of  those,  13  students  agreed  to  be  participants  in  the  study.  These  participants  were 
graduate  students  mainly  in  a program  in  educational  technology  which  was  the  program  that  hosted  the  course. 

Both  iterations  had  a small  number  of  students  from  other  fields.  About  three  quarters  of  the  students  were  located  in 
the  vicinity  of  the  educational  institution  during  both  implementations.  Flowever,  this  did  not  increase  reliance  on 
F2F  encounters  to  undergo  the  process  according  to  student  reports.  Students  had  very  limited  experience 
participating  in  online  collaborative  PBL. 

The  data  collected  for  this  study  consisted  of  the  analysis  of  online  documents  (e.g.  discussion  board 
postings,  chat  transcripts,  wikis,  emails),  administration  of  student  surveys,  and  instructor  interviews.  Both  student 
surveys  and  the  instructor  interviews  were  conducted  at  the  end  of  each  implementation.  The  analysis  of  documents 
spanned  the  duration  of  the  online  experience.  This  analysis  helped  devise  the  questions  for  surveys  and  interviews 
so  that  they  would  capture  important  areas  for  further  exploration  from  the  perspective  of  students  and  the  instructor. 
Data  was  analyzed  by  reading  through  it  one  time  to  identify  emerging  themes.  A second  reading  allowed  for  the 
coding  of  data  and  the  refinement  of  theme  categories.  Important  themes  that  were  identified  had  to  do  with  the  time 
allocated  for  activities,  the  strategies  students  used  during  the  process,  the  tools  that  supported  work,  their  feelings 
during  each  stage,  and  the  role  of  facilitators. 

The  researcher  took  several  measures  in  order  to  ensure  the  quality  of  the  investigation.  She  employed 
long-term  engagement  with  the  data  to  gain  a deep  understanding  of  the  process  (i.e.  one  year  in  understanding  the 
concept  of  online  collaborative  PBL,  one  year  undergoing  the  two  implementations  and  one  year  analyzing  the  data 
and  writing  up  the  report).  The  report  provided  rich,  thick  description  of  the  model  of  online  collaborative  PBL 
adopted  and  the  implementation  experience  viewed  from  the  eyes  of  the  researcher,  the  students,  and  the  course 
instructor.  The  study  used  a variety  of  sources  of  data  (i.e.  documents,  instructor  and  students’  perspectives)  and  a 
variety  of  methods  to  mine  the  data  (i.e.  analysis  of  such  documents,  surveys,  and  interviews).  The  implementation 
of  these  techniques  had  the  purpose  of  enhancing  internal  validity  and  reliability.  The  study  only  sought  external 
validity  by  allowing  readers  to  establish  the  relevance  of  these  findings  to  their  contexts  using  the  thorough 
descriptions  given. 

The  Evolution  of  the  Process 


Introduction  of  the  Problem 

The  introduction  of  the  problem  early  during  the  learning  experience  as  suggested  by  PBL  literature  was 
one  of  the  things  that  seemed  interesting  to  explore  in  the  online  milieu.  While  generally  students  did  not  think  the 
introduction  of  the  problem  had  been  too  early,  they  voiced  concerns  related  to  their  lack  of  knowledge  of  the  course 
requirements  and  their  peers  in  order  to  be  able  to  settle  for  problem  ideas  and  partners.  Students  who  appreciated  an 
early  introduction  valued  the  opportunity  to  start  their  thinking  about  it  early  on.  Thus,  further  research  on  this 
aspect  can  help  enlighten  ways  to  support  students  in  gaining  knowledge  about  the  course  requirements,  peers  and 
potential  problem  ideas  in  a variety  of  ways  so  that  this  design  consideration  can  be  implemented  in  online  contexts 
as  it  was  devised  for  F2F  implementations. 

Group  Formation 

The  process  of  group  formation  was  another  interesting  topic  in  the  context  of  this  research  study.  The  way 
this  process  occurred  in  the  two  semesters  together  with  the  differences  that  became  apparent  after  design  changes 
suggested  potential  areas  for  further  study.  Because  this  stage  took  a long  period  of  time,  it  is  necessary  to  study  a 
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combination  of  strategies  for  students  to  publicly  access  information  about  each  other  and  exchange  ideas  conducing 
to  forming  groups  early  on.  The  use  of  customizable  student  homepage  tools  is  recommended  so  that  course 
facilitators  request  that  students  answer  questions  about  themselves  that  are  highly  relevant  to  the  purposes  of  the 
project  instead  of  answering  generic  questions  about  themselves.  The  potential  value  of  synchronous  conversations 
can  be  studied  as  it  might  have  been  one  of  the  factors  by  means  of  which  students  in  Semester  1 were  able  to  form 
groups  slightly  more  quickly  enabling  additional  time  to  work  on  later  stages. 

There  were  variations  in  the  ways  students  went  about  the  task  of  forming  groups.  Semester  1 students 
conducted  synchronous  conversations  together  with  asynchronous  and  supplemented  by  reading  about  classmates  in 
the  CMS’  student  homepages.  The  potential  impact  of  the  emphasis  in  synchronicity  at  this  stage  (with  its  related 
challenges)  together  with  concerns  of  inability  to  be  synchronous  from  students  in  Semester  2 led  to  deemphasizing 
this  as  a strategy  for  group  formation.  Students  in  Semester  1 tended  to  use  synchronicity  very  often  at  the  beginning 
stages  while  relying  on  a bit  more  asynchronous  exchanges  towards  the  end.  Students  in  Semester  2,  on  the  other 
hand,  tended  to  work  asynchronously  but  voiced  concerns  later  on  related  to  the  potential  need  to  conduct 
synchronous  exchanges.  Sclater  & Bolander  (2004)  pointed  out  the  impact  of  the  introduction  of  an  activity  in  the 
value  that  students  place  on  conducting  it.  Other  authors  talk  about  humans  need  for  proximity  in  communication 
(Boden  & Molotch,  1994).  While  no  causal  relationships  can  be  established  because  of  the  nature  of  this 
investigation,  further  study  can  help  establish  ways  to  balance  the  need  of  immediacy  with  the  need  of  flexibility  so 
that  students  benefit  the  most  of  their  online  collaborative  PBL  experiences. 

The  Problem  Resolution  Process 

The  analysis  of  interactions  revealed  that  students  underwent  this  experience  in  a weave  of 
synchronous/asynchronous  exchanges,  brainstorming/negotiating/synthesizing  efforts  and  individual/collaborative 
co-authoring  of  group  documents.  Further  study  is  recommended  to  help  refine  this  model  of  approaches  to  conduct 
online  collaborative  PBL  to  see  whether  all  intersections  are  possible,  whether  it  fully  characterizes  the  process  and 
whether  any  particular  combination  is  more  likely  to  happen  at  particular  stages.  The  individual/collaborative 
continuum  has  been  explored  by  authors  like  Solomon  (1993,  as  cited  in  Hewitt  & Scardamalia,  1996)  with  his 
concepts  of  shared  and  off-loaded  cognition.  Understanding  this  can  help  select  the  most  appropriate  tools  to  support 
the  tasks  at  hand. 

The  adoption  of  strategies  seemed  influenced  by  factors  other  than  the  appropriateness  to  support  the  tasks 
at  hand.  Available  tools  seemed  to  drive  adoption  of  strategies.  The  ubiquity  of  email,  availability  of  students,  and 
limitations  of  tools  seemed  to  supersede  the  tools  match  to  support  the  tasks  at  hand.  This  selection  of  tools  also 
seemed  to  lead  students  more  often  in  the  direction  of  more  individual  work.  Sclater  & Bolander  (2004)  suggested 
that  the  factors  that  orient  students  towards  collaboration  include  previous  educational  experience,  deep  and  surface 
learning,  and  the  expectancy-value  motivation  theory.  While  these  factors  and  the  ones  in  this  study  do  not  concur,  it 
is  possible  that  the  factors  these  authors  mention  contribute  to  student  orientation  towards  collaboration  as 
characteristics  that  are  present  before  the  learning  experience  begins.  The  factors  identified  in  this  study  encourage 
different  modalities  of  collaboration  once  the  learning  experience  in  underway.  Despite  this  difference  of  factors,  the 
behavior  of  groups  in  co-authoring  documents  was  consistent  with  the  definitions  of  Sclater  & Bolander  (2004). 
They  characterized  some  of  the  groups  as  “cutting  down  and  stitching  together”  (individual  work  that  is  later 
integrated)  and  “negotiated  collaboration”  (weaving  contributions  by  an  elected  scribe).  These  two  characterizations 
are  compatible  with  the  findings  in  this  study. 

Together  with  strategies  for  learning,  online  collaborative  PBL  also  requires  strategies  for  the  management 
of  the  process.  Thus,  students  needed  to  coordinate  their  efforts,  assign  tasks,  report  on  their  progress  in  achieving 
the  tasks  and  schedule  synchronous  conversations  when  necessary.  The  depiction  of  these  tasks  is  consistent  with 
relevant  literature  on  the  nature  of  these  management  interactions  (Guribye,  Andreassen,  & Wasson,  2003).  These 
authors  stress  the  need  to  differentiate  computer-supported  collaborative  learning  from  computer-supported 
collaborative  work  when  studying  interaction.  Among  the  reasons  they  provide  for  this  differentiation  are  the  fact 
that  workers  know  their  roles,  form  more  stable  groups  while  students  are  novices  struggling  with  identifying  the 
roles  and  the  tasks  that  are  associated  with  them  and  form  more  temporary  groups  for  specific  educational  purposes. 
The  importance  of  understanding  collaboration  in  educational  settings  helps  avoid  borrowing  design  considerations 
from  fields  that  are  not  fully  compatible.  It  also  helps  to  develop  an  understanding  of  the  kinds  of  tools  that  are 
needed  to  support  educational  endeavors  to  make  progress  in  the  development  of  such  tools  (Garrison,  Anderson,  & 
Archer,  2003). 
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In  studying  the  use  of  tools  throughout  the  process,  it  was  evident  that  email  was  the  tool  that  accompanied 
students  every  step  of  the  way.  The  chat  complemented  email  for  students  in  Semester  1 and  the  wiki  complemented 
email  in  Semester  2.  Email  was  used  for  management,  group  formation,  role/task  assignment,  and  brainstorming  and 
negotiation.  In  some  cases,  it  was  deliberately  chosen  over  other  tools  that  could  have  better  supported  the  process. 
However,  its  use  proved  challenging  for  a number  of  reasons.  The  default  organization  of  email  inboxes  challenges 
the  support  of  the  process  as  it  buries  emails  within  the  list  of  other  personal  messages.  Prescribing  ways  of 
organizing  inboxes  to  separate  project  emails  would  entail  providing  specific  guidelines  about  the  use  of  the  tools. 
Nevertheless,  inboxes  that  reach  beyond  their  quota  can  interrupt  conversations  among  group  members  in  a way  that 
can  prove  unhealthy  for  the  well  being  of  the  group.  Finally,  using  email  for  such  tasks  as  scheduling  chat  sessions 
was  inefficient. 

While  email  provides  convenience,  chats  provide  a sense  of  immediacy.  Regardless,  the  use  of  chat  is  not 
exempt  from  difficulties.  Technical  problems  can  interfere  with  communication  thus  making  students  feel 
uncomfortable  when  they  are  unable  to  contribute  to  the  discourse  of  the  group.  The  use  of  free  providers  can 
complicate  matters  if  they  inundate  students’  email  accounts  with  spam  as  a result  of  the  “free  subscription”. 
Misunderstandings  can  be  exacerbated  online  sometimes  through  chat  even  in  spite  of  the  sense  of  immediacy.  In 
light  of  these  limitations,  it  is  necessary  to  study  ways  to  provide  for  immediacy  while  minimizing  the  weaknesses 
of  this  kind  of  supporting  technology  for  online  collaborative  PBL. 

Course  facilitators  adopted  the  wiki  to  provide  for  a place  to  hold  the  cumulative  knowledge  of  students 
(Hannafin,  Oliver,  Hill,  & Glazer,  2003).  The  wiki  tool  used  did  not  afford  contextual  identification  of  contributors 
thus  forcing  students  to  rely  on  other  conventions  to  allow  for  this.  However,  contextual  identifiers  can  be 
troublesome  even  when  only  two  contributors  edit  the  same  document.  This  can  be  furthered  studied  in  research  and 
development.  The  wiki  had  limitations  as  a tool  to  support  directly  composing  on  it.  The  particular  tool  that  was 
used  did  not  have  auto-save  features  and  constantly  saving  changes  impacts  the  history  tab  which  is  an  option 
(though  limited)  to  identify  contributors  and  their  changes.  It  did  not  have  the  advanced  editing  features  that  users 
are  accustomed  to  encounter  in  word  processing  programs.  Thus  students  resorted  to  compose  their  documents  in 
such  programs  and  paste  them  into  the  wiki.  This  brought  difficulties  in  that  the  formatting  was  altered;  even 
formatting  that  was  left  as  sign  posts  of  peers’  comments.  The  analysis  of  the  use  of  the  wiki  throughout  this 
experience  uncovered  a few  potentially  fruitful  pathways  for  further  study.  It  would  be  interesting  to  study  whether 
the  sense  of  common  grounds  it  provides  is  grounded  in  real  common  foundations  developed  by  group  members  or 
in  perceived  ones  that  might  be  nonexistent.  The  observation  of  the  variation  of  work  strategies  after  introducing 
different  tools  is  compatible  with  research  that  attributes  such  impact  to  tools.  Instructional  designers  of  online 
collaborative  PBL  also  need  to  consider  the  various  types  of  manifestations  of  cumulative  knowledge  that  they  will 
need  to  support  for  these  kinds  of  learning  processes.  The  wiki  can  help  support  the  co-authoring  of  mostly  written 
documents  but  can  fall  short  of  supporting  more  presentational  kinds  of  documents. 

In  considering  the  use  of  tools  to  support  online  collaborative  PBL,  two  more  aspects  are  worth  exploring. 
As  previously  stated  elsewhere  in  literature,  the  properties  of  supporting  media  need  to  be  carefully  matched  with 
activities  because  of  each  medium’s  affordances  and  constraints.  Guidelines  can  be  offered  so  students  make 
optimal  use  of  these  technologies  but  this  would  suggest  interference  with  the  self-directed  goal  of  PBL.  Most 
important,  however,  is  the  fragmented  support  that  these  technologies  offer  which  could  be  interfering  with  a 
knowledge  construction  process  that  is  intended  to  yield  a single  collaborative  artifact  that  evidenced  the  cumulative 
knowledge  of  the  group.  Many  tools  provide  support  for  synchronous/asynchronous,  individual/collaborative, 
brainstorming/negotiating/synthesizing  work.  However,  few  incorporate  support  for  all  those  kinds  of  interactions 
that  could  potentially  be  present  in  a semester-long  online  collaborative  PBL  process  together  with  support  for  the 
management  of  all  these  interactions.  The  study  of  tools  to  support  this  process  can  help  develop  tools  that  support 
all  work  strategies  in  efficient  ways.  The  current  trend  of  adoption  can  also  be  further  explored  to  identify  whether 
significant  pieces  of  the  cumulative  knowledge  of  the  group  are  lost  during  the  evolution  of  the  process  because  of 
the  inefficiency  of  this  fragmented  support. 

During  this  online  collaborative  PBL  experience  students  valued  the  active  participation  of  facilitators 
during  the  process.  The  value  was  consistent  with  the  way  groups  chose  to  work.  Thus,  students  who  engaged  in 
synchronous  conversations  valued  when  facilitators  participated  in  those  while  students  who  engaged  in  more 
asynchronous  work  valued  when  facilitators  provided  feedback.  Also,  feedback  was  more  valuable  in  later  stages 
when  more  asynchronous  work  superseded  synchronous  work.  This  valuing  is  consistent  with  the  views  of  teaching 
presence  in  literature  and  its  impact  in  student  learning  and  motivation  (Swan,  2003).  In  providing  feedback, 
facilitators  were,  on  the  one  hand,  contributing  to  students  perception  of  future  success  and,  on  the  other  hand, 
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enabling  them  to  improve  their  processes  and  products  along  the  way  (Sclater  & Bolander,  2004;  Zumbach  & 
Reimann,  2003).  Students’  value  of  structural  support  was  also  evident  in  their  appraisal  of  the  use  of  milestones 
throughout  the  process. 

There  is  a tension  between  the  self-directed  goal  of  PBL  (Barrows,  1986),  adult  learning  theory  (Merriam 
& Caffarella,  1999)  and  students’  need  for  more  structure  as  evidenced  in  this  study.  Authors  in  literature  also  speak 
of  increased  need  for  structure  online  (Steinkuehler,  Derry,  Hmelo-Silver,  & DelMarcelle,  2002).  Some  recommend 
a number  of  ways  in  which  guidance  can  help  scaffold  online  collaboration.  Swan  (2003)  has  spoken  about  the 
greater  attention  to  detail  that  is  necessary.  In  describing  efforts  to  support  online  communities,  Kling  & Courtright 
(2004)  suggest  an  interventionist  strategy  in  moderating  online  group  work.  Finally,  Rummel  & Spada  (2005)  found 
that  more  guided  collaboration  through  scripts  or  examples  shows  positive  effects  on  processes  and  outcomes  in 
online  student  collaboration.  Because  of  these  issues,  it  is  important  to  further  research  to  understand  how  to  support 
students  in  self-directed  PBL  online  in  light  of  their  needs  for  more  structure  in  these  contexts. 

On  average,  about  half  of  the  students  (51%)  expressed  positive  feelings  throughout  the  stages  of  the  online 
collaborative  PBL  process.  Students  in  Semester  1 decreased  in  their  expression  of  positive  feelings  as  the  process 
evolved,  however.  Moreover,  students  in  Semester  2 expressed  less  positive  feelings  during  their  first  stage.  A 
number  of  potential  reasons  can  be  further  explored  in  causing  this.  The  whole  group  and  small-groups  in  semester  1 
were  larger.  This  results  in  more  work  during  interactions.  Students  in  Semester  2 had  more  time  to  devote  to  the 
process  because  of  regular  calendar  dates.  Students  in  Semester  1 did  not  have  a tool  to  hold  the  cumulative 
knowledge  of  the  group.  Further  research  can  help  enlighten  whether  smaller  whole  and  small-groups  can  be  more 
efficient  in  this  context.  It  can  also  help  elucidate  whether  the  choice  of  tools  to  support  the  process  can  help  foster  a 
more  promotive  atmosphere. 

Students  more  often  reported  that  what  impacted  their  feelings  during  the  process  was  the  nature  of  the 
groups’  dynamics.  Authors  in  literature  explain  that  social-emotional  needs  of  group  exist  and  need  to  be  superseded 
so  they  can  move  on  to  focus  on  more  task-related  needs.  While  the  original  PBL  model  recommends  that  students 
engage  in  group  processing,  students  in  this  study  continually  reported  difficulties  in  gauging  their  peers’  reactions 
to  their  ideas  thus  presenting  a potential  challenge  to  online  group  processing.  Emoticons  can  help  here  but  they  are 
dependent  on  whether  the  system  supports  them  and  whether  students  know  the  full  range  options  available  to 
express  their  feelings.  Course  facilitators  did  not  structure  this  as  it  was  left  to  the  self-directedness  of  students. 
However,  in  implementing  these  guidelines  of  PBL,  it  might  be  necessary  to  explore  whether  specific  guidelines 
need  to  be  given  to  students,  and  how  to  enable  this  in  light  of  the  limitations  of  supporting  technologies. 

Ownership  of  the  problem  is  important  in  PBL  (Savery  & Duffy,  1996).  In  light  of  this,  students  were 
allowed  to  propose  their  own  ideas  for  problems  to  solve.  However,  this  brought  about  different  levels  of  sense  of 
ownership  among  students.  This  affected  whether  they  felt  comfortable  in  suggesting  certain  kinds  of  courses  of 
action.  Thus,  further  study  can  help  enlighten  whether  high  levels  of  ownership  can  still  be  fostered  while  using 
generic  problems  in  areas  that  are  likely  to  interest  students. 

While  agreement  with  a statement  that  time  allocation  had  been  sufficient  was  much  higher  in  Semester  2 
than  in  Semester  1,  about  80%  of  students  on  average  agreed  that  time  was  sufficient  throughout  the  stages. 

Semester  2 students  had  more  time  to  undergo  this  process  so  it  is  likely  to  see  this  as  the  cause  for  their  higher 
agreement.  This  has  implications  for  the  length  of  PBL  experiences  online.  It  suggests  that  further  research  informs 
whether  these  long-term  experiences  are  optimal  or  whether  shorter  processes  are  equally  optimal  when  a long-term 
all-encompassing  process  is  not  possible. 

Conclusions 

In  speaking  about  the  limitations  of  conducting  online  collaborative  PBL,  students  pointed  out  a number  of 
reasons:  impact  of  mediated  communication,  longer  time  to  conduct  communications,  difficulties  in  scheduling 
collaboration  events  when  using  multiple  perspectives  of  students  geographically  dispersed  and  group  dynamics 
aggravated  online.  However,  a large  percentage  of  students  in  both  semesters  pointed  out  it  had  been  a beneficial 
experience  for  them.  The  reasons  for  this  benefit  included  establishing  a clear  link  between  theory  and  practice, 
meaningful  learning  in  context  (because  of  the  particular  case  of  this  course  that  focuses  on  CMC),  flexibility  and 
convenience,  the  benefit  of  reflection,  archiving,  access  to  collaborate  with  dispersed  peers,  learning  to  be  more 
articulated,  the  positive  side  of  the  lack  of  physical  cues,  and  more  time  on  task. 

This  research  helped  confirm  statements  of  other  authors  in  literature  that  state  that  online  collaborative 
PBL  is  possible  (Orrill,  2002;  Price,  2000;  Shoenfeld-Tacher,  McConnell,  & Kogan,  2004).  In  advancing  the 
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understanding  of  this  methodology  when  it  happens  online,  this  research  study  has  helped  uncover  areas  that  are 
fruitful  for  further  inquiry.  This  way,  educational  institutions  that  have  taken  on  the  goal  of  providing  meaningful 
learning  to  their  online  students  of  education  can  do  so  and  thus  help  them  learn  how  to  promote  these  kinds  of 
constructivist  learning  environments  for  their  own  students. 
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Introduction 

Hundreds  of  international  Educational/Instructional  Technology  (ET/IT)  graduates  enter  the  field  each  year.  This 
paper  examines  their  progression;  where  they  go,  what  jobs  they  accept,  how  long  they  stay;and  if  they  leave,  where 
do  they  go?  Can  we  learn  how  to  help  international  students  climb  corporate  or  academic  ladders?  What  barriers  did 
they  experience  and  can  these  be  eliminated  or  reduced?  Can  American  ET/IT  programs  help  international  students 
have  quality  experiences  and  fulfilling  employment  progressions? 

Background,  Justification,  and  Purpose 

On  any  given  autumn  day  at  innumerable  airports  across  America,  hundreds  of  international  students  arrive  ready  to 
begin  their  studies  in  Educational/Instructional  Technology  graduate  programs.  If  we  fast  forward  between  that  time 
and  two-to-seven  years,  the  reverse  process  repeats  itself  as  we  wave  farewell  to  our  cherished  colleagues  and 
friends.  In  many  instances,  we  stay  in  touch  with  them,  at  least  for  the  coming  year  or  so,  just  to  make  sure  they 
arrived  at  their  destination  safely  and  went  to  an  acceptable  job.  Then  what?  That  is  what  we  will  explore. 

Were  we  to  chronologically  arrange  the  arrival  and  departure  of  international  students  to  and  from  America  the 
litany  would  reveal  that  international  scholars  have  been  coming  to  our  shores  for  the  past  two  centuries  or  more 
from  every  corner  of  the  globe.  Our  task,  rather  than  catalog  the  comings  and  goings  of  these  thousands,  is  restricted 
to  only  those  AECT  members  or  members  of  its  precedent  organizational  variants,  most  of  whom  were  engaged  in 
the  study  of  instructional  technology  in  one  form  or  another. 

We  wanted  to  find  out  who  some  of  these  individuals  were  and,  while  we  had  the  entire  world  to  use  as 
database,  decided  to  concentrate  only  on  East  Asia,  more  specifically,  on  China  and  Taiwan.  We  did  this  as  a matter 
of  convenience  rather  than  for  any  specific  empirical  reasoning,  because  it  is  a source  already  well-known  to  us. 

We  wanted  to  trace  the  progression  of  a selected  number  of  Asian  graduate  students  from  the  time  they  entered  an 
American  university  to  the  present.  We  wanted  to  track  their  career  travels  and  look  at  both  the  places  they  went  as 
well  as  learn  something  about  their  academic  qualifications.  Going  on  the  axiom  that  history  repeats  itself,  we 
wanted  to  identify  the  histories  of  this  select  group  of  graduates  in  an  effort  to  determine  their  own  perceptions  of 
achieved  success.  In  doing  so  we  wanted  to  benefit  from  their  perceptions  of  how  life  had  treated  them  along  the 
way,  and  to  ascertain  instances  when  things  had  not  necessarily  gone  as  well  as  they  had  hoped. 

We  wanted  to  identify  ups,  downs,  and  in-betweens  of  their  careers,  to  plot  where  they  had  been,  where  they  went, 
and  how  long  they  tarried  at  each  point  along  the  way.  We  wanted  qualitative  responses  from  them  as  to  what  each 
event  meant  to  them  and  those  impacted  by  their  presence.  We  wanted  to  get  their  impressions  of  what  went  right, 
what  did  not,  what  had  best  prepared  them  for  their  future  careers  and  what  was  lacking  that,  had  they  known  better, 
would  have  altered  their  course. 

Method 

Following  along  the  lines  of  a questionnaire  developed  by  the  authors:  C.  C.  Pan,  Y.  Tao,  Z.  Wang,  R.  A.  Cornell, 

C.  H.  Ku,  C.  H.  Lee,  and  H.  Y.  Ku  (2001),  we  modified  that  instrument  to  obtain  answers  to  the  following  ten 
questions  from  1 1 participants: 

1.  When  did  you  come  to  the  United  States  for  your  first  graduate  study  and  where  was  it?  1950-60 

1960-70 1970-80 1980-90  1990-2006 Location  by  state  or  country: 
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2. 


What  was  the  terminal  degree  received,  what  was  your  major,  and  when  was  it  awarded?  M.A. 
M.S. Ed.D PhD Major: Year: 

3.  What  were  the  most  significant  (or  useful)  courses  you  took  during  that  timeframe? 

4.  What  were  the  least  significant  (or  useful)  courses  you  took  during  the  same  timeframe? 

5.  What  were  the  useful  courses  that  you  wish  you  had  taken  during  the  same  timeframe? 

6.  What  was  your  first  job  upon  graduation;  where  was  it,  and  how  long  did  you  remain  at  that 

location? 

7.  Can  you  list  the  subsequent  jobs,  titles,  locations  and  durations? 

8.  What  have  been  major  highlights  in  your  career? 

9.  What  have  been  difficult  times  (job  related)  in  your  career? 

10.  What  would  you  do  differently,  given  the  passage  of  time  that  would  help  an  emerging  young 
Asian  scholar  to  fulfill  his  or  her  career  dreams? 

In  responding  to  the  first  three  questions,  surveyed  individuals  reported  the  following  demographics: 

• The  majority  (over  80%)  of  the  respondents  came  to  the  United  States  during  the  past  ten  to  15 
years. 

• Approximately  one  third  received  the  first  graduate  degree  in  Florida.  Another  one  third  went  to  a 
university  in  other  parts  of  the  United  States. 

• More  than  80%  were  awarded  a doctoral  degree  (mostly  Ph.D.). 

• Over  80%  received  a terminal  degree  in  Educational/Instructional  Technology. 

• The  majority  (over  80%)  graduated  between  2000  and  2005. 

Invitations  were  emailed  to  major  Chinese  Educational  Technology  groups  and  organizations,  such  as  International 
Division  of  the  AECT  and  the  Society  of  International  Chinese  in  Educational  Technology.  Data  were  collected  via 
email  and  they  were  complied  and  analyzed  accordingly. 

Results  and  Findings 

Questions  3 through  10  were  used  to  support  the  bulk  of  this  data  collection  effort.  Each  of  the  questions  is  listed 
below,  followed  by  the  preliminary  results. 

3.  What  were  the  most  significant  (or  useful)  courses  you  took  during  that  timeframe? 

Approximately  half  of  the  participants  reported  some  type  of  Instructional  Systems  Development  course  was  the 
most  useful  course.  The  second  significant  course  was  a research  course  related  to  educational/instructional 
technology. 

4.  What  were  the  least  significant  (or  useful)  courses  you  took  during  the  same  timeframe? 

Over  half  of  the  respondents  seemed  quite  satisfied  with  the  curriculum.  To  name  one  least  significant  course,  these 
participants  chose  some  Education  foundations  course  that  is  often  considered  a Ph.D.  in  Education  program  core 
course. 

5.  What  were  the  useful  courses  that  you  wish  you  had  taken  (but  did  not)  during  the  same  time  frame? 

About  60%  of  the  surveyed  individuals  wished  they  had  taken  courses  in  some  special  topics  in  Educational 
Technology,  such  as  Human  Performance  Technology  (HPT),  electronic  learning  (eLearning),  and  course 
management  systems/learning  management  systems  (CMS/LMS). 

6.  What  was  your  first  job  upon  graduation;  where  was  it,  and  how  long  did  you  remain  at  that  location? 

All  the  Ph.D./Ed. D.  graduates  worked  at  the  higher  education  on  the  first  job.  The  majority  who  stayed  after 
graduating  had  a United  States  employer.  A majority  appeared  to  change  their  job  or  move  to  a higher  ranked 
position  within  five  years. 

7.  Can  you  list  the  subsequent  jobs,  titles,  locations  and  durations? 

For  those  who  stayed  in  the  U.S.,  their  subsequent  jobs  were  scattered  across  the  United  States.  For  those  who 
returned  to  their  home  country,  they  tended  to  seek  employment  in  prestigious  institutions  (e.g.,  by  a national 
university)  due  to  perceived  increased  job  security. 

8.  What  have  been  major  highlights  in  your  career? 

Because  a majority  (over  80%)  of  this  select  group  works  in  higher  education,  teaching  and  research  appeared  to  be 
major  highlights  in  their  career  lives.  Two  (male)  of  the  participants  were  able  to  serve  as  administrators  of  some 
note  in  their  institution  (Taiwan). 

9.  What  have  been  difficult  times  (job  related)  in  your  career? 

When  it  comes  to  “work  slump,”  those  surveyed  reported  employment  difficulties  (about  30%),  foreign  cultural 
differences  (about  20%),  and  marrying  ISD  to  another  discipline  (about  20%)  have  posed  difficult  times  in  their 
careers. 

10.  What  would  you  do  differently,  given  the  passage  of  time  that  would  help  an  emerging  young  Asian  scholar  to 
fulfill  his  or  her  career  dreams? 
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Given  a second  chance,  40%  of  them  would  seek  professionalism  by  choosing  a specific  area  and  stick  to  it  all  the 
way  through  while  in  school.  Another  40%  would  manage  to  collaborate  more  with  professors  and  colleagues  on 
research  and  development  projects  before  graduation.  About  30%  wished  that  they  had  demonstrated  good  English 
language  proficiency  before  starting  to  work.  Two  of  the  respondents  (about  20%)  emphasized  the  importance  of 
mentorship,  and  they  recommended  that  new  Asian  scholars  find  a way  to  approach  the  faculty  advisor/mentor  and 
learn  from/with  him  or  her. 

Summary,  Discussions,  and  Further  Questions 

Each  year  hundreds  of  students  from  China  and  Taiwan  attend  and  graduate  from  an  ET/IT  graduate  program  in  the 
United  States.  Their  job  progressions  and  perceptions  of  the  program  curriculum  are  two  primary  foci  of  this 
working  paper.  This  investigation  should  interest  those  newly  arrived  Chinese  students  and  concern  our  colleagues 
and  friends  who  teach  in  an  ET/IT  or  related  program  in  the  States.  The  results  and  findings  are  intended  to 
empower  the  newcomers  and  prepare  them  for  a more  productive  academic  career  than  their  predecessors. 

The  qualitative  study  was  conducted  using  a purposive  sample.  The  data  were  solicited  via  email  and  analyzed  using 
hand-coded  content  analysis. 

Its  preliminary  results  indicated  that  all  of  our  international  students  graduated  and  worked  in  the  related  area  in  the 
United  States,  whether  or  not  they  received  a terminal  degree.  However,  most  of  them  received  a doctoral  degree 
and  stayed  in  the  higher  education.  The  design  (a.k.a.  Instructional  System  Development)  course  and  research  in 
Instructional  Technology  are  two  of  the  most  useful  courses.  Education  (as  opposed  to  ET/IT)  foundations  courses 
appeared  unfavorable.  These  respondents  also  recommended  that  special  topic  courses  that  best  represent  the 
evolving  nature  of  ET/IT,  such  as  Human  Performance  Technology,  eLearning,  and  CMS/LMS  be  included  in  the 
study  plan.  There  seemed  to  be  a pattern  that  the  new  graduates  changed  jobs  within  the  five  years  after  the 
graduation.  These  international  students  would  go  almost  anywhere  in  the  country  to  seek  employment.  They 
enjoyed  research  and  development  wherever  they  were  employed.  Regarding  the  work  slump  they  have 
encountered,  some  seemed  to  have  a hard  time  getting  employed  when  they  first  graduated  and  some  found 
difficulty  in  blending  into  the  American  culture.  These  young  scholars  came  up  with  a list  of  recommendations  for 
those  who  will  follow.  Their  recommendations  were: 

• Pursue  professionalism.  Choose  a distinct  area  in  ET/IT  earlier  and  make  effort  to  become  an 
expert  in  that  area. 

• Value  collaboration.  Don’t  sacrifice  research  and  development  opportunities  with  colleagues  and 
faculty  n order  to  graduate  early.  Such  collaborative  experiences  may  speak  highly  on  the  resume. 

• Strengthen  language  proficiency.  Acquiring  English  language  skills  and  knowing  more  about  the 
American  culture  can  render  the  transition  and/or  adjustment  to  the  new  environment  much  easier. 

Teaching  classes  can  be  very  helpful  in  this  regard. 

• Enhance  networking.  The  ET/IT  field  is  a small  world  in  the  sense  that  most  people  know  each 
other  and  they  communicate  often  with  each  other.  Staying  in  touch  with  your  mentor  and  keeping  your 
friends  close  can  easily  expand  your  horizons  of  knowledge  about  the  field.  This  informal  relationship 
usually  affords  frequent  opportunities  for  cross-organizational  research  and  development  projects. 

From  these  responses  we  hope  that  graduate  students,  be  they  Asian  or  not,  will  have  gained  more  insight  as  to  how 
careers  unfold,  how  individuals  deal  with  situations  that  arise,  and  ultimately,  how  they  fulfill  their  own 
expectations.  Further  questions  below  will  be  appended  to  this  article  and  conclusions  drawn. 

1 . Are  the  career  experiences  of  these  individuals  similar  or  different  than  those  that  have  come  from 
other  Asian  countries? 

2.  Are  there  similarities  or  differences  that  might  be  anticipated  by  graduates  from  countries  outside 
of  Asia,  i.e.,  Latin  America,  Europe,  Africa,  Australia,  India? 

3.  If  I am  currently  in  an  American  instructional  technology  graduate  program  and  getting  ready  to 
graduate  and  accept  my  first  post-degree  employment,  what  can  I take  away  from  this  session  that  I can  use 
tomorrow? 

4.  Does  coming  from  a different  culture  make  me  more  or  less  attractive  as  a job  applicant  within  the 
United  States? 

5.  By  returning  to  my  home  country  and  taking  my  first  job  after  having  graduated,  will  my 
American  degree  help  or  hinder  my  prospects  for  advancement? 
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Abstract 

This  paper  reviews  key  elements  that  contribute  to  the  successful  integration  of  technologies  and  constructivist 
pedagogical  activities  in  K12  settings.  Theoretical  constructs  of  constructivism  were  identified  from  the  relevant 
literature  such  as  (a)  preferred  theoretical  perspective,  (b)  the  common  notion  of  crisis  in  learning  mechanism,  and 
(c)  methodological  concerns.  These  theoretical  constructs  framed  a review  of  ten  selected  empirical  studies  on  the 
use  of  technology  in  classrooms  that  published  for  the  period  of  1999  to  2006.  The  main  finding  suggested  that 
insufficient  elaboration  of  constructivist  theoretical  constructs  in  empirical  studies  resulted  in  poor  results  or  no 
significant  effects  on  constructivist  learning. 


Introduction 

I still  feel  like  I’m  teaching  things  the  same  way  I used  to. 

(A  secondary  teacher.  Interview,  December  2004) 

A great  deal  of  research  has  examined  teachers’  adaptations  of  Information  and  Communication  Technologies 
(ICT)  into  the  classroom  (e.g.,  Blumenfeld,  Fishman,  Krajcik,  Marx,  & Soloway,  2000;  Bracewell,  Le  Maistre, 
Lajoie,  & Breuleux,  in  press;  Bryson  & de  Castell,  1998;  Windschitl  & Sahl,  2002).  This  research  suggests  that  ICT 
is  a pedagogical  mediator  that  promotes  constructivist  or  student-centered  classroom  learning.  However,  in  contrast 
to  the  theoretical  findings  of  educational  researchers,  teachers  in  most  classrooms  continue  to  struggle  with  teaching 
practices  that  have  barely  changed  since  they  began  adopting  technologies  into  their  classrooms.  As  Windschitl 
(2002)  argued,  teachers  face  various  pedagogical  challenges  in  integrating  ICT  into  instruction  including  facilitating 
student  knowledge  construction  and  collaboration,  distributing  knowledge  and  roles  and  evaluating  learning. 

This  paper  reviews  key  elements  that  contribute  to  both  the  successful  and  unsuccessful  integration  of  ICT  and 
constructivist  pedagogical  activities  into  K-12  settings  that  have  been  reported  in  the  literature.  Theoretical 
constructs  of  constructivism  are  identified  based  on  literature  about  constructivist  theories.  These  theoretical 
findings  in  turn  frame  a review  of  recent  empirical  studies  of  technology-supported  learning  environments.  This 
review  aims  to  identify  key  elements  in  the  integration  of  ICT  into  constructivist  classrooms  based  on  differences 
and  similarities  between  theoretical  constructs  and  empirical  findings.  Finally,  we  will  suggest  some  implications  of 
the  findings  for  designing  teacher  education  programs. 

Constructivism  as  a Theory 

Human  knowledge  is  constructed. 

(Phillips,  1995,  p.5) 

Although  we  do  not  believe  anymore  that  human  knowledge  is  simply  absorbed  through  sensory  experiences  or  is 
transmitted  from  one  knower  to  another,  it  has  been  only  two  decades  since  we  began  seeing  knowledge  as  by-and- 
large  constructed.  Despite  the  relative  newness  of  this  emerging  view  of  knowledge,  constructivism  has  become  akin 
to  a religion  in  the  field  of  education.  My  intention  in  making  an  analogy  between  “religion”  and  constructivism, 
which  was  inspired  by  Phillips  (1995),  is  not  to  exaggerate  constructivism’s  impact  on  education.  Rather,  the 
analogy  is  entirely  apt  for  two  reasons;  first,  just  as  a religion  has  its  own  folk-tales  explaining  human  origins, 
constructivism  has  its  own  account  of  the  origins  of  knowledge;  and  second,  as  with  many  living  religions, 
constructivism  has  many  rival  sects  within  it. 

In  this  section,  we  will  discuss  first  how  constructivism  has  altered  our  view  of  the  origin  and  nature  of  human 
knowledge.  Second,  we  will  illustrate  the  many  faces  of  constructivism  in  education,  laying  stress  on  two  prominent 
constructivist  perspectives:  radical  constructivism  and  socio  constructivism. 
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Nature  of  Knowledge 

How  can  we  come  to  know  anything  about  our  world?  How  can  we  approach  knowledge  that  can  contribute  to  its 
growth?  Epistemology  is  the  discipline  that  seeks  answers  to  questions  about  the  nature  and  justification  of  human 
knowledge  (Hofer  & Pintrich,  1997).  Different  paradigms  in  education  often  reflect  different  epistemological 
traditions.  Each  paradigm  has  its  own  beliefs  about  how  one  comes  to  know  and  what  contributes  to  the  growth  of 
knowledge.  When  a paradigm  shifts,  its  prevailing  epistemological  perspective  often  changes  accordingly  in  a way 
to  inform  and  justify  educational  practices  (Gergen,  1995). 

The  predominant  paradigm  in  the  field  of  education  from  the  1950’s  through  the  1970’s  was  behaviorism,  where 
the  perspectives  on  knowledge  stem  from  a positivist  epistemology.  Positivism  held  knowledge  to  consist  of  “hard 
facts”  that  can  be  observed,  empirically  tested,  and  proved  or  disproved.  Because  positivists  held  that  right  answers 
must  exist  “out  there”,  knowledge  in  behaviorism  was  seen  as  something  that  can  be  transmitted  from  one  knower  to 
another  and  reinforced  by  stimulus-and-response.  Drawing  on  such  an  epistemology,  Skinner,  a prominent 
behaviorist,  saw  learning  as  “an  experimental  analysis  of  behavior”  (Skinner,  1974).  In  other  words,  learning 
outcomes  are  a series  of  repeated  behaviors  reinforced  by  external  stimuli,  and  the  internal  causes  of  such  behavioral 
changes  are  of  no  interest  in  this  paradigm.  Skinner’s  notion  of  a “black  box”  in  learners  was  associated  with  an 
unexplained  mechanism  of  how  one  comes  to  know.  Because  of  this  theoretical  deficit,  behaviorism  failed  to  remain 
in  a leading  position  in  the  field  of  education. 

Unlike  behaviorism,  constructivism  per  se  is  a kind  of  epistemology.  Constructivism’s  view  of  how  learning  occurs 
is  based  on  an  understanding  of  “how  conceptual  development  should  be  approached  and  how  it  could  be  fostered” 
(von  Glasersfeld,  1995a,  p.5).  It  was  to  meet  the  growing  awareness  of  a need  for  alternatives  to  the  traditional 
theories  of  knowing  in  Western  philosophy,  which  were  unable  to  sufficiently  explain  how  one  becomes  a knower. 

Constructivism  views  knowledge  as  theory-laden  and  is  philosophically  influenced  by  interpretivism  or 
postmodernism.  This  means  that  users  interpret  knowledge  differently  and  that  knowers  construct  new  knowledge 
based  on  prior  knowledge  and  experiences. 

Since  the  time  that  constructivism  became  the  leading  paradigm  in  education,  educational  efforts  have  focused  on 
understanding  and  enhancing  the  processes  by  which  learners  construct  their  knowledge.  In  doing  so,  some  theorists, 
such  as  Piagetians  and  radical  constructivists,  attribute  knowledge  construction  to  the  cognitive  structures  within 
individual  learners;  while  others,  such  as  Vygotskian  or  social  constructivists  attribute  knowledge  construction  to 
social  interaction  between  learners  or  between  learners  and  environments  involving  the  sharing  and  negotiating  of 
meaning  and  culture. 

In  following  section,  we  will  illustrate  and  compare  these  two  constructivist  perspectives,  the  main  sects  of 
constructivism  in  education. 

Radical  Constructivism 

Since  Piaget  (1970)  claimed  that  knowing  is  an  individual  adaptive  activity,  it  has  been  an  unfalsifiable  claim  in 
education  that  students  construct  their  own  knowledge.  Piagetian  constructivism,  which  is  often  referred  to  as  radical 
constructivism,  argues  that  learning  occurs  through  a process  of  disequilibrium,  where  learners  constantly  eliminate 
perturbations  by  accommodating  new  concepts  or  operations  that  prove  incompatible  with  existing  knowledge.  That 
is,  cognitive  change  or  learning  takes  place  when  use  of  a scheme  leads  to  perturbation,  and  then  to  an 
accommodation  to  the  scheme  in  order  to  maintain  or  re-establish  equilibrium  (von  Glasersfled,  1995a).  To  Piaget, 
therefore,  cognitive  development  is  regarded  as  “expanding  equilibration”,  an  increase  in  the  range  of  perturbations 
that  the  organism  is  able  to  eliminate  in  much  the  same  way  as  organisms  biologically  adapt  themselves  to  survive  in 
ever-changing  environments. 

It  is  important  to  note  that  Piaget’s  notion  of  action  schemes  is  radically  different  from  the  behaviorists’  notion  of 
stimulus-response.  Although  action  schemes  also  underlie  the  feedback  loops  that  cause  changes  in  action,  action 
schemes  are  explicitly  goal-directed.  This  goal-directedness  of  action  scheme  enables  learners  to  become 
increasingly  self-regulated  through  the  mechanisms  of  assimilation  and  accommodation. 

The  term,  “radical”  constructivism,  associated  with  Piaget’s  genetic  epistemology,  was  first  introduced  by  Smock 
and  von  Glasersfled  in  1974  in  their  research  report,  “The  implications  of  radical  constructivism  for  knowledge 
acquisition”.  As  a researcher  in  the  field  of  scientific  reasoning,  von  Glasersfeld  argues  that  “the  virtually  undisputed 
domination  of  a mindless  behaviorism”  (1995b,  p.4)  for  half  of  the  20th  century  confounded  the  distinctions  between 
training  (for  performance)  and  teaching  (for  understanding).  This  was  an  unfortunate  consequence  of  least  two 
factors.  First,  science  was  seen  as  a way  to  absolute  truth;  and  second,  therefore,  a drill-and-practice  type  of 
reinforcement  training  was  believed  to  lead  learners  to  the  achievement  of  intelligent  behavior.  However,  when  von 
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Glasersfeld  found  students  in  his  school  who  became  unable  to  solve  problems  that  call  for  deeper  understanding 
than  mere  rote-memorization,  he  began  seeking  a new  perspective  of  learning  that  would  be  able  to  inform  how 
conceptual  development  should  be  approached.  Radical  constructivism,  in  this  regard,  was  an  alternative  approach 
that  saw  knowledge  as  “the  internal  mental  construction  of  the  individual”  (Smith,  1995,  p.  23). 

Associated  with  cognitive  psychology,  research  programs  driven  by  radical  constructivism  have  flourished  in 
generating  theories  of  teaching  and  learning.  This  is  especially  the  case  in  the  fields  of  science  and  math  education 
where  researchers  have  focused  on  understanding  the  knowledge  structures  that  experts  construct  and  on  modeling 
efficient  access  to  and  retrieval  of  relevant  knowledge  for  solving  problems  (e.g.  Ericsson  & Simon,  1984;  Newell  & 
Simon,  1972;  Ericsson  & Smith,  1991).  While  studying  and  externalizing  these  cognitive  processes,  researchers 
have  become  aware  of  how  to  scaffold  or  facilitate  student  construction  of  knowledge. 

Elowever,  radical  constructivism  has  also  been  criticized  for  its  negligence  of  social  interaction  during  the  course  of 
knowledge  construction.  Paour  (1990)  argues  that  it  is  because  Piagetian  constructivism  “seeks  to  locate  the 
psychological  mechanism  of  adaptation  in  direct  continuity  with  biological  mechanisms”  (p.  178).  Phillips  (1995) 
also  criticizes  the  fact  that  von  Glasersfeld’s  epistemology  is  developed  in  a way  that  is  flawed  in  much  the  same 
way  as  positivist  epistemology  is  flawed.  To  Phillips,  radical  constructivism  is  not  a sufficient  theory  of  knowledge 
construction  or  learning  because  it  ignores  social  factors  and  it  supports  a standard  set  or  structure  of  correct 
conceptions  that  all  learners  should  have.  Socio-constructivism  has  emerged  to  meet  the  demands  for  a more 
extended  consideration  of  social  and  cultural  influence  on  knowledge  construction. 

Socio-Constructivism 

Unlike  radical  constructivism,  socio-constructivism  has  many  sects,  which  has  perhaps  contributed  to  complaints 
about  theoretical  illusion  in  constructivism.  In  their  book,  “ Construction  in  Education” , Steffe  and  Gale  (1995) 
categorize  socio-constructivism  into  three  different  paradigms:  social  constructionism,  social  constructivism,  and 
socio-cultural  approaches  to  mediated  action. 

The  best  representatives  of  social  constructionism  might  be  Thomas  Kuhn  (1962)  and  Imre  Lakatos  (1970). 
Although  they  have  opposing  perspectives  regarding  how  scientific  knowledge  grows,  both  stress  the  active  role  of 
scientific  communities  in  knowledge  construction.  Since  social  constructionism  places  greater  emphasis  on 
theorizing  the  social  dimensions  of  knowledge  construction  than  on  education,  we  will  not  deal  with  it  any  further  in 
this  paper. 

Both  social  constructivism  and  the  socio-cultural  approach  are  rooted  in  Vygotsky’s  accounts  of  social  aspects  of 
learning.  Differences  between  these  two  approaches  reside  in  their  focus  of  inquiry.  That  is,  the  former  focuses  on 
individual  development  through  support  or  interaction  with  others  whereas  the  latter  focuses  on  collective  efforts 
mediated  by  cultural  means  including  material  and  immaterial  artifacts  in  everyday  activities. 

Social  constructivism  employs  Vygotsky’s  (1978)  notion  of  the  “zone  of  proximal  development”  (ZPD).  To  social 
constructivists,  cognitive  development  is  analogous  to  expanding  one’s  ZPD.  The  distances  between  levels  of  actual 
and  potential  development  are  determined  by  proper  guidance  by  experts  (adults)  or  peers.  Thus  children  can  widen 
their  ZPD  infinitely  in  talking  and  sharing  with  others.  “Cognitive  apprenticeship”  (Collins,  Brown,  & Newman, 
1989)  and  “reciprocal  teaching”  (Brown  and  Campione,  1994)  are  two  well-known  instructional  developments  of 
the  notion  of  ZPD. 

However,  more  radical  socio-cultural  perspectives  criticize  socio-constructivism  for  its  insufficient  account  of  what 
drives  learning  and  human  transformations  in  social  and  historical  contexts.  From  socio-cultural  perspectives, 
mental  functioning  cannot  be  understood  without  considering  social  contexts,  including  participation  in  social 
settings  and  the  historical  development  of  cultural  artifacts  and  tools  (Wertsch  & Toma,  1995). 

In  fact,  it  was  Leont’ev  (1978),  Vygotsky’s  student  and  colleague,  who  expanded  the  notion  of  mediation  from 
individuals  to  collective  activities.  This  expansion  called  for  a better  account  for  collective  nature  of  activities;  and  it 
was  Engestrom  who  later  elaborated  Leon’ev’s  concepts  into  his  Activity  Systems  Theory  (AST)  by  adding  such 
theoretical  constructs  as  rules,  communities  and  the  division  of  labor  in  order  to  better  represent  dynamic  mediation 
in  collective  activities  (Engestrom,  1987).  By  adding  these  constructs,  Engestrom  distinguished  AST  from  several 
other  versions  of  activity  theory  in  at  least  two  ways:  object-relatedness  of  mediation  (Stensenko,  2005)  and 
cultural-historical  views  for  organizational  transformation. 

In  Engestrom’s  (1999)  construct  of  “Expansive  Cycles  of  Learning  Action”  learning  occurs  in  crisis  situations 
when  members  of  an  organization  share  common  goals  of  overcoming  the  crises.  When  existing  routines  become 
problematic,  members  of  an  organization  begin  to  externalize  problems  and  to  seek  and  to  test  new  models  to 
replace  existing  ones  (by  either  creating  new  cultural  means  or  artifacts  or  by  adjusting  existing  ones,  such  as  rules, 
documents,  technology,  etc).  Consequently,  a new  model  is  chosen  and  becomes  internalized  and  routinized  by 
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members,  and  finally  the  crisis  is  resolved.  In  the  course  of  these  collective  activities,  members  learn  and  these 
learning  processes  are  again  historically  accumulated  in  their  organization. 

However,  socio-constructivism  cannot  avoid  the  criticism  of  overemphasizing  external  activities  in  social  setting 
and  underemphasizing  individual  knowledge  structures  and  cognitive  processes,  which  cannot  be  isolated  from 
external  actions.  Radical  constructivists  find  the  notion  of  artifacts  that  are  believed  to  historically  mediate  activities 
in  socio-constructivism  especially  unacceptable.  Von  Glasersfeld  (1989)  argues  that  one  can  never  know  what  is  in 
mind  of  another  person  by  reading  a book  or  looking  at  other  artifacts  because  the  meaning  conveyed  by  words  and 
other  linguistic  vehicles  are  inescapably  subjective  for  both  producers  and  receivers. 

Summary 

In  the  above  section,  we  examined  theoretical  origins  and  constructs  of  constructivism  and  their  relevance  to  the 
nature  of  knowledge.  A summary  of  the  examination  is  provided  in  Table  1. 

Based  on  a review  of  constructivist  theories,  with  particular  emphasis  on  how  the  nature  of  knowledge  has  been 
considered  in  radical  constructivism  and  socio  constructivism,  three  major  framing  considerations  were  identified: 
(a)  preferred  theoretical  perspective,  (b)  the  common  notion  of  crisis  in  learning  mechanism  and  (c)  methodological 
concerns. 


Table  1.  A summary  of  theoretical  constructs  in  constructivism 


Sects 

Representatives 

Learning  Principles 

Methodology 

Domains 

Constructivism 

Radical 

Piaget 

Von  Glasersfeld 

Equilibrium  between  existing 
and  new  knowledge  through 
iterative  process  of 
accommodation  and 
assimilation 

Modeling 

knowledge  structure 
(i.e.  Expertise 
approaches) 

Mathematics 

Medical 

education 

(reasoning 

processes) 

Socio 

Social 

constructivism 

Brown,  A. 
Brown,  J. 
Lave 

Social  interaction 
Shared  cognition 
Cognitive  apprenticeship, 
LPP 

Discourse  analysis 

Conversation 

analysis 

Science  education 

(project-based 

learning) 

Literacy 

education 

Socio-cultural 

approaches 

Vygotsky 

Leont’ev 

Engestrom 

Mediated  learning 
Expansive  learning  cycle  in 
crisis 

Experimental 
treatment 
Discourse  analysis 
Phenomenol-ogical 
studies 

Cognitive/ 

social 

development 

Workplace 

learning 

First,  constructivist  perspectives  differ  with  respect  to  the  nature  of  knowledge  depending  on  the  discipline  under 
consideration  (Cobb,  1994;  Bereiter,  1994).  Thus,  the  process  of  formulating  research  questions  place  researchers 
within  particular  perspectives  because  perspectives  on  knowledge  construction  in  literacy  education  often  differ 
from  those  in  science  education.  For  example,  one  the  one  hand,  researchers  in  the  fields  of  mathematics 
(Schoenfeld,  2002)  and  medical  education  (Kaufman  & Patel,  1991;  Patel  & Groen,  1986)  prefer  radical 
constructivism  since  they  tend  to  focus  on  learners’  reasoning  processes.  On  the  other  hand,  in  science  education 
where  project-based  learning  is  commonly  used  researchers  prefer  socio-constructivism  as  they  tend  to  focus  on 
social  influences  on  collaborative  knowledge  construction. 

Second,  as  a learning  principle,  the  constructs  of  expanding  equilibrium  in  Piagetian  (radical)  constructivism 
(Piaget,  1970;  von  Glasersfeld,  1995a)  and  the  construct  of  expansive  learning  cycles  (Engestrom,  1999)  in 
Vygotskian  (socio)  constructivism  (Vygotsky,  1978)  are  basically  alike  with  respect  to  the  importance  of  “crisis”  for 
learning.  The  only  difference  between  the  two  perspectives  is  that  the  former  sees  crisis  as  “internal  occurrence 
within  an  individual”  whereas  the  latter  sees  crises  as  “organizational  and  cultural  occurrences  in  a community”. 

Both  perspectives  require  learning  tasks  to  be  designed  so  as  to  enable  students  to  critically  appraise  their  existing 
knowledge  and  to  externalize  and  reconcile  latent  discrepancies  in  their  knowledge. 

Third,  together  with  the  previous  two  findings,  different  theoretical  stances  alter  the  methodologies  to  be  employed. 
For  example,  on  the  one  hand,  if  one  wants  to  investigate  the  knowledge  structure  and  reasoning  processes  of  an 
expert  mathematician,  drawing  on  radical  constructivism,  cognitive  protocol  analysis  with  the  provision  of  well- 
designed  tasks  is  more  appropriate.  On  the  other  hand,  in  order  to  systematically  examine  social  and  individual 
phenomena  associated  with  the  adaptation  of  new  practices  such  as  constructivist  activities  by  teachers,  it  is  more 
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appropriate  for  researchers  to  adopt  a socio  cultural  approach  such  as  the  activity-theoretical  framework. 

These  theoretical  findings  will  frame  a review  of  empirical  studies  on  technology-supported  constructivist 
classrooms,  which  will  be  presented  in  the  following  section. 

Constructivism  in  practice:  A review  of  empirical  studies 

The  socio-constructivist  environment  in  which  students  were  confronted 
might  have  been  too  complex  to  be  handled  constructively 
(Clarebout  and  Elen,  2001,  p.  462) 

So  far,  we  have  examined  the  theoretical  constructs  of  constructivism  in  terms  of  the  similarities  and  differences 
between  two  mainstream  constructivist  perspectives.  Based  on  the  preceding  theoretical  review,  we  will  review 
recent  empirical  studies  on  technology-supported  constructivist-oriented  learning  environments.  The  purpose  of  this 
review  is  to  find  out  how  well  the  theoretical  constructs  have  been  reflected  on  in  those  studies.  The  goal  is  to 
examine  my  hypothesis  that  current  problems  in  the  unsuccessful  use  of  ICT  reside  in  insufficient  reflection  of  the 
theories  of  constructivism,  and  thus  in  the  inadequate  design  of  methodologies. 

To  select  articles  to  be  reviewed,  ERIC  and  Psychlnfo  databases  were  searched  with  the  keywords  “constructivis? 
and  technology  and  (elementary  or  secondary)”.  The  search  was  limited  to  journal  articles  published  in  the  period  of 
1999  to  2006.  Articles  on  higher  education  were  removed  from  the  search.  The  database  generated  totaled  54  articles. 
Then,  articles  dealing  with  distance  education  and  problems  caused  by  poor  access  to  technology,  as  well  as  opinion 
papers  rather  than  empirical  studies,  were  removed.  The  final  set  consisted  of  10  articles.  A thumbnail  of  the  selected 
articles  is  presented  in  Table  2. 

The  review  was  carried  out  based  on  the  theoretical  findings  in  the  previous  section,  such  as  preferred  theoretical 
perspective  with  respect  to  discipline  under  consideration,  the  common  notion  of  crisis  in  learning,  and 
methodological  concerns.  Findings  suggested  that  there  are  three  critical  discrepancies  between  theoretical 
constructs  and  their  actual  application  in  empirical  studies;  (a)  ambiguous  theoretical  stances  of  most  of  the 
reviewed  studies,  (b)  as  a result,  inappropriate  study  design  for  the  nature  of  domains,  and  (c)  insufficient  reflection 
on  theoretical  constructs  of  constructivism  with  respect  to  the  tasks  under  investigation.  In  what  follows,  we  will 
present  these  three  findings  in  more  detail. 

Ambiguous  Theoretical  Stances  and  Constructs 

Among  the  ten  articles,  only  three  (Churach  et  ah,  2001;  Clarebout  et  ah,  2001;  Lavonen  et  ah,  2002)  explicitly 
claim  their  theoretical  stances  to  be  either  radical  or  socio  constructivism.  For  the  rest  of  the  articles,  we  inferred 
their  theoretical  tendencies  based  on  a close  review.  For  example,  although  Ioannidou  et  ah  (2004)  and  Vincent 
(2001)  commonly  employed  Papert’s  (1993)  LOGO  program,  the  former  used  it  for  group  projects  whereas  the  latter 
for  individual  learning.  Therefore,  we  inferred  the  former  to  have  adopted  a socio  constructivist  perspective  the  latter 
to  have  adopted  a radical  constructivist  perspective. 

This  is  not  to  say  that  educational  researchers  must  explicitly  choose  a constructivist  perspective  for  their  inquiry. 

In  fact,  the  need  for  balanced  considerations  between  the  two  perspectives  has  been  widely  recognized  by  both 
camps  of  constructivism  (Phillips,  1995;  Cobb,  1994;  Bereiter,  1994;  Longino,  1993).  Phillips  (1995)  in  particular 
criticizes  the  continued  existence  of  a persistent  distrust  between  the  two  camps  as  such  one-sided  view  is 
unwelcome  in  pursuing  the  growth  of  theories  of  learning.  We  cannot  ignore  individual  cognitive  changes  while 
actively  engaging  in  learning  activities  nor  can  we  neglect  the  surrounding  natural  contexts  that  place  considerable 
constraints  on  knowledge  construction  and  allow  us  to  detect  our  errors. 

In  this  regard,  Windschitl  and  Sahl’s  (2002)  article  presents  a nicely  balanced  constructivist  theoretical  perspective. 
The  authors  used  the  five  characteristics  of  constructivist  classrooms  originally  developed  by  Becker  and  Ravitz 
(1999)  to  frame  their  inquiry.  These  elements  included  (a)  having  students  engage  in  collaborative  work,  (b) 
designing  activities  around  teacher  and  student  interests,  (c)  focusing  on  student  understanding  of  complex  problems 
and  ideas,  (d)  assessing  their  understanding,  and  (e)  (teachers’)  engaging  in  learning.  These  characteristics  provided 
the  authors  with  a trajectory  that  adequately  blends  the  two  constructive  perspectives  and  with  a lens  to  trace  how 
technology  transforms  teaching  and  learning  practices. 
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Table  2.  Outline  of  reviewed  articles 


Authors 

Discipline/ 

Technology 

used 

Participants 

Theoretical 
framework/cons  tracts 

Methodology 

Identified  factors  for 
success  (if  any) 

Churach  & 
Fisher  (2001) 

Science/ 
Internet  and 
email 

431  secondary 
students  in 
Hawaii 

Radical  constructivism 
(explicit) 

CLES  survey  (by 
Taylor  and  Fraser, 
1991)  and  classroom 
observation 

Active  internet  usage 
and  teachers’ 
supportive  attitude 

Clarebout  & 
Elen  (2001) 

Multidisciplina- 
ry / Intranet 
between  six 
schools 

124  secondary 
students  in 
Europe 

Socio-constructivism 

(explicit) 

Pre-post  test 

Proper  and  controlled 
implementation  of 
the  designed 
environment 
Difficulties  of  tasks 

Ioannidou  et 
al.  (2004) 

Science/ 

Computer 

simulations 

4th  and  5th 
graders 

Social  constructivism 

(project-based 

simulations) 

Case  study 
(Protocol  analysis  of 
audiotaped  classroom 
discourse) 

Redefined 
curriculum  in 
accordance  with 
constructivist 
learning  principles 

Lavonen  et 
al.  (2002) 

Science  / 
Computer 
(programming 
tool) 

Thirty  four  8th 
graders 

Socio  constructivism 
(explicit;  collaborative 
problem  solving) 

Case  study 
(Videotaped  protocol, 
critical  incident 
analysis) 

Teacher  support 

Rakes  et  al. 
(1999) 

Not  particular/ 
Not  particular 

435  randomly 
sampled  K12 
teachers 

Radical  constructivism 

Survey 

Professional 

development 

Rice  et  al. 
(2001) 

Social  studies/ 
All  available 
technologies  at 
the  school 

A secondary 
teacher 

Unclear 

Case  study 

Belief  of 

administrators  and 
parents 

Roberts 

(2004) 

Literacy/ 

Virtual 

communities 

35  students  in 
Wyoming  and 
15  in  Costa  Rica 

Socio  constructivism 
(Project-based  learning  in 
a virtual  community) 

Case  study 

Common  language 
for  students  to  use 

Vincent 

(2001) 

Writing/ 

Computer 

(MicroWorld) 

Five  5th  graders 
in  Melbourne 

Radical  constructivism 
(Papert’s  LOGO,  but 
unclear) 

Case  study 

Wallace  et  al. 
(2000) 

Science  / 
Internet 

Eight  6th 
graders  in  a 
middle  school  in 
US 

Socio  constructivism 

Case  study 

Windschitl  et 
al.  (2002) 

Multi  domains 
(social  studies 
and  math)  / 
Laptop 

Three  teachers 
at  a middle 
school  in  US 

Hybrid  (explicitly  stated 
in  five  elements  of 
constructivist  classrooms) 

Case  study  using  an 

ethnographic 

perspective 

It  is  important  to  distinguish  between  balanced  theoretical  considerations  and  ambiguous  theoretical  stances.  For 
example,  one  article  (Rice  et  ah,  2001)  had  such  unclear  perspectives  on  constructivism  that  it  was  impossible  to 
infer  a clear  constructivist  theoretical  framework.  More  seriously,  a few  studies  in  the  set  did  explicitly  present  their 
theoretical  stance  either  as  radical  or  socio  constructivism;  however,  their  actual  focus  of  inquiry  did  not  reflect  the 
theoretical  framework  they  claimed  to  have  adopted.  For  example,  Clarebout  et  al  (2001)  explicitly  claimed  socio 
constructivism  as  their  theoretical  framework;  however,  in  the  study  the  actual  constructs  that  they  measured  were 
the  development  of  meta-cognitive  strategies.  This  is  not  to  say  that  what  they  looked  at  was  totally  inappropriate. 
Indeed,  social  interaction  supported  by  network  technologies  can  promote  the  development  of  student  meta- 
cognitive  strategies.  However,  unclear  explanation  on  how  authors  get  from  networked  technologies  to  meta- 
cognitive  strategies  is  problematic. 

A clear  theoretical  stance  might  not  be  so  critical  for  practitioners  such  as  teachers  when  promoting  constructivist 
activities  in  their  classroom;  however,  researchers  have  a responsibility  to  provide  practitioners  with  rigorous 
theoretical  bases  in  promoting  such  activities. 
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Inappropriate  or  Defective  Methodology’ 

As  a result  of  their  ambiguous  theoretical  stances,  inappropriate  study  designs  were  observed.  These  included 
inadequate  methodologies  for  the  nature  of  the  domain  that  studies  looked  at  and  the  failure  to  position  technologies 
as  a construct  in  the  studies. 

In  this  set  of  articles,  a case  study  method  was  observed  in  seven  studies  out  of  ten.  Quantitative  methods  using 
surveys  and  paper-pencil  pre-post  tests  were  employed  in  two  of  the  studies  and  the  rest  one  used  both  survey  and 
follow-up  observations  and  interviews. 

Case  study  methodology  is  accepted  as  an  appropriate  methodology  for  investigating  questions  concerning  the  how 
and  why  of  contemporary  phenomenon  in  real  life  contexts  (Yin,  2003).  Because  there  are  countless  factors  and 
incidences  to  consider  in  real  life  settings,  a salient  framework  and  thorough  analysis  informed  by  a suitable 
theoretical  framework  is  mandatory  for  supporting  the  findings  of  such  research.  However,  this  was  not  the  case 
with  the  ways  in  which  the  case  studies  reviewed  here  were  carried  out.  For  example,  Roberts  (2004)  addressed  so 
many  issues  in  her  literature  review  (e.g.  technology  in  education,  constructivism,  technology  accessibility,  foreign 
language  learning  and  multiculturalism,  etc.)  that  it  is  ambiguous  as  to  what  theoretical  framework  informed  her 
study.  Consequently,  it  is  difficult  to  determine  what  constructs  she  used  to  examine  the  pedagogical  activities  in 
this  learning  environment  and  what  framework  she  used  to  analyze  her  data.  Unsurprisingly,  the  unit  of  analysis  was 
also  unclear.  Although  Roberts  (2004)  mentioned  “case  data”  (Stenhouse,  1978)  as  a unit  of  analysis,  she  analyzed, 
or  more  accurately  described,  interview  data  instead  of  cases  that  students  produced. 

Another  problem  with  the  case  studies  is  that  only  one  article  (Lavonen  et  al.,  2002)  reported  inter-rater  reliability 
with  respect  to  the  data  coding  and  none  of  them  conducted  a member  check.  Consequently,  it  is  questionable 
whether  interview  or  observation  data  were  properly  and  faithfully  analyzed  or  the  author(s)  simply  counted  on  data 
that  seemed  to  support  their  hypotheses.  For  example.  Rice  et  al.  (2001)  analyzed  a massive  amount  of  data  that  they 
had  collected  for  over  a five-year  period  including  e-mail  interviews,  participants’  course  assignments,  classroom 
observations,  course  plans,  and  so  on  and  generated  four  categories  using  constant  comparative  analysis.  In  doing  so, 
they  reported  no  more  than  the  following: 

To  produce  an  accurate  presentation  of  the  research  findings,  as  well  as  to  control  for  researcher  bias,  data 
were  triangulated  across  the  research  team  and  data  sources,  (p . 2 1 5 ) 

It  is  more  important  to  describe  how  they  triangulated  the  data  than  to  simply  mention  what  they  did  to  increase  the 
trustworthiness  of  the  research. 

The  quantitative  studies  in  this  set  also  have  methodological  problems.  For  example,  Clarebout  (2001)  did  a factor 
analysis  to  identify  constructs  to  include  in  each  of  the  four  scales:  epistemological  beliefs,  instructional  belief, 
meta-cognitive  strategies,  and  task-related  knowledge.  However,  it  is  unknown  from  which  theoretical  framework 

these  scales  drew  on.  Moreover,  the  reliability  of  the  scales  (Cronbach  a)  for  assessing  epistemological  beliefs  and 
meta-cognitive  strategies  were  only  .56  and  .62  respectively,  which  are  quite  low.  Not  surprisingly,  this  study 
produced  poor  results.  That  is,  after  participating  in  the  project,  students’  epistemological  and  instructional  beliefs, 
meta-cognitive  strategies,  and  task-related  knowledge  declined.  Although  the  authors  blamed  the  poor  results  on  the 
implementation  stage  rather  than  the  design  stage,  they  are  more  likely  to  be  a consequence  of  the  poor  scales  that 
did  not  measure  what  they  were  supposed  to. 

Designed  Learning  Environments 

Some  of  the  “constructivist”  learning  environments  that  the  studies  investigated  were  purposely  designed,  ranging 
from  innovative  curricula  to  computer  simulation  design  programs  and  huge  networks  between  schools.  However, 
some  of  the  design  principles  that  they  applied  were  appeared  to  be  inappropriate.  Constructivist  learning  tasks  do 
not  imply  that  students  are  allowed  to  construct  everything  for  their  learning.  Rather,  they  imply  that  learning  tasks 
be  more  carefully  designed  to  appraise  learners’  pre-existing  knowledge,  and  more  importantly,  to  model  and 
scaffold  activity  in  order  to  help  students  construct  their  own  learning  path. 

In  this  regard,  Ioannidou  et  al.  (2004)  provided  one  of  the  few  good  examples  of  well-designed  learning  tasks  in 
this  set.  The  authors  investigated  how  5th  and  6th  graders  model  and  understand  the  ecosystem  using  a computer 
simulation  program.  The  authors  asked  students  to: 

design  imaginary  animals  which  do  not  actually  exist  but  which  could  live  in  their  environment.  The  animal 
design  must  include  adaptations  for  the  animal  to  survive  the  temperatures  common  to  its  environment  as 
well  as  to  mate  and  to  acquire  food.  Within  each  group,  students  collaboratively  work  out  the  predator-prey 
relationships  for  their  ecosystems,  (p.  70) 
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With  clear  guidance  for  creating  their  own  “eco-world”,  students  were  so  engaged  in  the  task  that  they  referred  to 
their  creature  in  the  first  person,  e.g.  “I  can  eat  you”  rather  than  “my  animal  can  eat  yours”.  Because  their 
assumptions  about  their  creatures  were  constantly  tested  in  the  simulation  and  resulted  in  a living  or  dead  ecosystem, 
this  learning  activity  helped  students  to  establish  constant  re-equilibrium  through  processes  of  accommodation  and 
assimilation. 

Unfortunately,  such  well-designed  tasks  were  not  always  provided  in  the  studies  reviewed.  For  example,  Vincent 
(2001)  stated: 

A constructionist  environment  had  been  fostered  by  the  participant  researcher  in  the  term  prior  to  the  study,  in 
particular  an  environment  in  which  children  were  encouraged  to  investigate  all  possibilities  for  use  of 
computers  in  the  learning  environment,  and  in  which  it  was  acceptable  to  set  problems  beyond  the  minimum 
tasks  proposed  by  the  teacher  (p.  244). 

This  principle  of  a constructivist  learning  environment  proposed  by  the  author  is  somewhat  misleading. 
Constructivist  learning  does  not  mean  that  students  try  to  find  their  way  in  using  learning  tools  or  materials  or  in 
performing  little  tasks  proposed  by  teachers.  Rather,  teachers  must  provide  students  with  well-designed  tasks  that 
enable  them  to  challenge  their  existing  knowledge  and  eventually  “expand  equilibration”  in  Piaget’s  term. 

Wallace  et  al.  (2000)  designed  learning  tasks  that  did  not  sufficiently  reflect  constructivist  learning  theories.  The 
authors  investigated  how  students  look  for  and  approach  science  information  on  the  web.  To  do  so,  the  authors  had 
students  generate  three  questions  that  they  wanted  to  investigate  on  the  web,  which  turned  out  to  be  problematic 
with  regard  to  learning  processes  as  well  as  research  itself.  Because  students  did  not  fully  recognize  the  goals  of 
such  activity,  they  stuck  to  the  original  questions  and  got  lost  rather  than  constantly  refining  their  questions  so  as  to 
become  more  relevant  with  respect  to  constructing  their  own  knowledge  around  what  they  were  interested  in.  As  a 
result,  students  focused  on  the  searching  phase  instead  of  on  all  the  stages  (e.g.  question  asking,  information 
gathering  and  evaluating,  and  information  synthesizing  and  using).  In  the  discussion,  the  authors  correctly  asserted 
that: 

It  confirms  research  on  task  that  tells  us  that  a complex  task  in  and  of  itself  does  not  cause  complex 
thinking. . ..It  matters  what  students  are  asked  to  do  and  how  tools  and  techniques  to  accomplish  these  tasks 
are  provided,  (p.97) 

In  sum,  few  of  the  studies  reviewed  investigated  learning  environments  that  were  designed  based  on  the  theoretical 
constructs  of  constructivism  such  as  the  notion  of  equilibrium  by  Piaget,  ZPD  by  Vygotsky,  or  expansive  learning 
cycle  by  Engestrom.  Together  with  ambiguous  theoretical  stances,  insufficient  reflection  on  theories  for  designing 
learning  environments  appears  to  be  a critical  factor  in  explaining  the  outcomes  of  studies.  Without  such  theory- 
based  principles  of  design,  constructivist  learning  environments  can  easily  become  merely  trendy  mumble  jumble 
fads  having  no  effects  on  student  knowledge  construction. 

Implications  for  the  Effective  Integration  of  ICT  into  Constructivist  Classrooms 

In  the  previous  sections,  we  examined  the  theoretical  constructs  of  constructivism  in  terms  of  radical  and  socio 
constructivism.  Also,  we  reviewed  ten  empirical  studies  to  understand  the  current  trends  in  the  use  of  ICT  to 
promote  constructivist  classrooms.  Based  on  differences  and  similarities  between  theoretical  constructs  and 
empirical  findings,  we  will  suggest  some  key  elements  in  the  successful  integration  of  ICT  into  constructivist 
classrooms. 

Task  Design  and  Motivation 

According  to  Piaget’s  notion  of  equilibrium,  it  is  important  to  provide  students  with  tasks  that  challenge  their  prior 
knowledge  scheme.  The  tasks  should  be  neither  too  easy  nor  too  difficult.  However,  in  technology-supported 
learning  environments,  there  can  be  two  different  challenges:  new  technology  environments  and  new  types  of 
learning  experience  or  tasks.  As  pointed  out  by  one  of  the  reviewed  studies  (Clarebout  et  al.,  2001),  it  might  be  too 
challenging  for  students  to  become  motivated  if  they  find  both  the  tasks  and  the  environments  to  be  unfamiliar. 
Therefore,  it  is  necessary  for  teachers  to  be  aware  of  the  importance  of  balance  between  degrees  of  task  difficulty 
and  student  motivation  in  designing  ICT  supported  constructivism  activities. 

Teachers  ’Attitude  and  Professional  Development 

Related  to  the  first  element,  teachers’  active  engagement  in  student  constructivist  learning  is  essential,  ranging  from 
the  careful  planning  and  designing  of  curriculum  and  activities  to  supportive  and  flexible  attitudes  toward  student 
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self-directed  learning.  This  adds  credibility  to  Bracewell  et  al.’s  (in  press)  findings  that  a successful  teacher  attitude 
was  detected  as  appropriate  “release  of  agency”.  The  research  indicates  that  one  of  the  critical  requirements  for 
successful  ICT  integration  into  a constructivist  classroom  is  that  teachers  must  be  comfortable  with  relinquishing 
authority  and  with  distributing  responsibilities  for  learning. 

Meanwhile,  cultural-historical  approaches  provide  an  adequate  methodology  for  investigating  teachers’ 
instructional  activities  associated  with  their  attitudes  and  beliefs.  In  fact,  there  has  been  an  increasing  number  of 
studies  employing  Activity  Systems  Theory,  one  of  cultural-historical  approaches  to  understand  how  teachers  use 
ICT  to  adapt,  abandon,  and  reinvent  new  practices  (e.g.  Bracewell,  Tung,  and  Sicilia,  2005;  Windschitl  and  Sahl, 
2002).  Such  research  contributes  to  the  effective  use  of  ICT  in  constructivist  classrooms  by  phenomenologically 
analyzing  how  members  of  such  organizations  including  teachers,  students,  parents,  staffs  and  administrators 
collectively  overcome  classroom  crises  arising  from  new  instructional  practices. 

Along  the  same  line,  such  research  contributes  to  more  systematic  instructional  design  in  professional  development. 
As  a participant  teacher  of  one  study  in  the  reviewed  set  stated,  it  is  challenging  for  teachers  to  conceive  of  “the 
intersection  between  the  technology,  the  curriculum,  and  classroom  management  without  knowing  what  a laptop- 
equipped  classroom  looked  like”  (Windschitl  and  Sahl,  2002,  p.  178).  A cultural-historical  approach  can  help 
instructional  designers  model  and  design  activity  systems  in  classrooms  that  take  into  consideration  multiple  goals 
and  participants,  the  nature  of  tasks,  the  uptake  of  tasks,  shifts  in  roles,  etc.,  which  are  socially,  culturally,  and 
pedagogically  more  complex  than  in  traditional  classrooms.  Such  modeled  activity  systems  in  turn  make  teachers 
aware  of  the  socio-cultural  contradictions  between  their  current  assumptions  and  practices  and  new  practices  needed 
for  the  effective  use  of  ICT  to  promote  constructivist  pedagogical  activities. 

Administrative  Support 

Finally,  the  data  or  findings  from  the  field  also  speak  to  the  need  for  theoretical  development.  Although  it  is  not 
mentioned  in  any  of  the  theories,  administrative  supports  are  a critical  factor  in  successfully  putting  constructivism 
into  practice.  This  is  an  obvious  example  for  showing  gaps  between  theory  and  practice.  In  other  words,  without  full 
understanding  of  nature  of  administrative  work  and  situations,  researchers'  enormous  efforts  to  design  and 
implement  constructivist  classrooms  with  a solid  theoretical  background  will  often  fail  to  get  off  the  ground 
(Windschitl,  2002).  Cultural-historical  approaches  provide  possible  solutions  to  the  problem  by  investigating 
existing  historical  divisions  of  labor  and  cultural  rules  among  members  of  communities  including  administrators  and 
teachers  in  order  to  understand  and  model  activity  systems  and  to  prevent  the  occurrence  of  possible  problems. 

Conclusions 

The  purpose  of  this  paper  was  to  review  key  elements  that  contribute  to  the  successful  and  unsuccessful  integration 
of  ICT  and  constructivist  pedagogical  activities  into  K-12  settings  that  have  been  reported  in  literature. 

Constructivist  theoretical  constructs  were  identified  based  on  literature  about  constructivist  theories.  These 
theoretical  findings  in  turn  framed  a review  of  recent  empirical  studies  on  technology-supported  learning 
environments  to  identify  key  elements  in  the  successful  and  unsuccessful  integration  of  ICT  in  constructivist 
classrooms. 

Based  on  a review  of  constructivist  theories,  with  particular  emphasis  on  how  the  nature  of  knowledge  has  been 
conceived  in  radical  constructivism  and  socio  constructivism,  three  major  framing  considerations  were  identified. 
First,  researchers’  perspectives  on  constructivism  differ  from  one  discipline  to  another  (Bereiter,  1994;  Cobb,  1994). 
Second,  as  a learning  principle,  the  construct  of  expanding  equilibrium  in  Piagetian  (radical)  constructivism  (Piaget, 
1970;  von  Glasersfeld,  1995)  and  that  of  the  expansive  learning  cycle  (Engestrom,  1999)  in  Vygotskian  (socio) 
constructivism  (Vygotsky,  1978)  are  basically  alike.  Both  imply  that  learning  tasks  should  be  designed  to  enable 
students  to  critically  appraise  their  existing  knowledge  and  to  externalize  latent  discrepancies.  Third,  together  with 
the  previous  two  findings,  different  theoretical  stances  were  found  to  alter  methodologies  to  be  employed. 

Given  these  theoretical  findings,  a set  of  empirical  studies  on  technology-supported  constructivist  classrooms  was 
reviewed.  The  articles  were  selected  based  on  keyword  searches  of  relevant  databases  and  consisted  of  ten  journal 
articles  from  the  period  of  1999  to  2006.  Findings  suggested  that  there  were  three  critical  discrepancies  between 
theoretical  constructs  in  theory  and  practice:  first,  most  of  the  reviewed  studies  exhibited  ambiguous  theoretical 
stances,  second,  as  a result  of  the  first  discrepancy  most  of  reviewed  studies  employed  inappropriate  designs  for  the 
nature  of  domains  investigated,  and  third,  most  of  the  studies  failed  to  adequately  reflect  constructivist  theoretical 
constructs  in  the  tasks  under  investigation.  Insufficient  elaboration  of  constructivist  theoretical  constructs  in 
empirical  studies  resulted  in  poor  results  or  no  significant  effects  on  constructivist  learning.  It  is  suggested  that  more 
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thorough  methodological  concerns  are  needed  to  produce  trustworthy  studies. 

The  findings  presented  three  implications  for  the  successful  integration  of  ICT  for  promoting  constructivist 
pedagogical  activities:  (a)  learning  tasks  should  be  designed  in  a way  to  constantly  challenge  students  pre-existing 
knowledge  with  the  help  of  teachers  and  other  peers;  (b)  teachers  need  to  be  more  active  in  planning  and  enacting 
classroom  activities.  It  can  be  possible  with  appropriate  professional  development  programs  that  draw  on  a more 
thorough  theoretical  framework  to  reveal  what  technology-supported  classrooms  look  like.  Both  principles  of  design 
for  constructivist  learning  experiences  and  awareness  of  possible  problems  likely  to  occur  in  constructivist 
classrooms  are  important  findings  in  terms  of  implications  for  what  to  include  and  how  to  design  professional 
development  programs.  Finally,  (c)  administrative  support  is  a critical  factor  in  the  successful  integration  of  ICT  into 
constructivist  classrooms. 

Research  must  inform  practice  and  at  the  same  time,  must  be  inspired  by  practice.  It  is  our  responsibility  as 
researchers  to  assist  and  guide  such  a teacher  who  “still  feels  like  s/he  is  teaching  things  the  same  way  s/he  used  to” 
to  do  their  job  better  in  such  new  and  innovative  learning  environments. 
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From  Lecture  into  Hands-on:  Improving  Instructional  Technology 
Workshops  for  Faculty  Development 
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Abstract 

In  Spring  2006,  the  Faculty  and  Curriculum  Support  Center  at  Georgetown  University  Medical  Center 
began  a faculty  development  workshop  improvement  project  to  convert  instructional  technology  workshops 
in  large-group  lecture  format  into  small-group  hands-on  workshops  to  allow  the  workshop  participants  to 
get  rich  learning  experiences  and  enhance  their  learning  outcomes.  This  presentation  explores  the 
conversion  of  the  workshop  format  and  reveals  and  evaluates  the  outcomes  of  the  project  one  semester 
later. 


Introduction 

Since  Spring  2003,  the  Faculty  and  Curriculum  Support  (FACS)  Center  at  the  Georgetown 
University  Medical  Center  has  provided  faculty  with  instructional  technology  workshops,  which  have 
taught  basic  skills  of  software  applications  for  web  and  multimedia  development  and  played  an  important 
role  in  faculty  development  in  the  medical  center.  For  the  first  two  semesters  since  the  workshops  began, 
the  primary  goal  of  the  workshops  was  to  make  as  many  faculty  members  as  possible  attend  the  workshops. 
The  workshops  were  held  in  a lecture  format  at  a classroom  with  20  stations.  The  software  applications 
were  installed  only  on  the  instructor’s  station  and  the  workshop  participants  just  watched  the  instructor’s 
presentation.  They  used  their  stations  only  for  viewing  example  files  and  browsing  web  sites  the  instructor 
presented.  Despite  recognizing  the  disadvantage  of  the  workshop  format  and  its  classroom  setup,  FACS 
held  the  workshops  at  the  classroom  to  accommodate  more  number  of  participants  at  a time.  This  made  the 
workshop  instructors  have  difficulty  in  making  the  workshops  fully  meet  the  participants’  expectations. 

After  the  workshops  have  been  fully  implemented  for  two  years,  FACS  has  started  to  concern 
about  the  quality  of  the  workshops  rather  than  the  number  of  workshop  participants.  From  the  workshop 
participants’  feedback  collected  for  last  two  years,  FACS  found  that  what  they  most  wanted  was  hands-on 
training,  which  would  have  allowed  them  to  apply  software  for  themselves  and  try  out  the  skills  they 
learned  during  the  workshop  sessions.  This  was  a very  critical  point  especially  for  teaching  workshops  on 
multimedia. 

FACS  had  a small  computer  lab  with  3 PC  stations  with  all  the  web  and  multimedia  applications 
for  the  workshops,  such  as  Photoshop,  Dreamweaver,  and  Flash,  and  used  the  lab  for  technology 
consultation,  one-on-one  training,  and  working  on  faculty  projects.  To  overcome  the  shortcoming  of  the 
lecture-format  workshops,  FACS  let  the  workshop  participants  know  it  would  provide  them  with  one-on- 
one  training  sessions  for  their  hands-on  learning  experience  at  the  computer  lab  after  the  workshops  on 
their  request.  This,  however,  did  not  work  as  designed.  Almost  no  one  signed  up  for  any  one-on-one 
sessions.  Even  though  they  could  get  hands-on  experience,  they  did  not  want  to  spend  another  time  to  learn 
the  same  skills  they  thought  they  had  already  acquired  at  the  workshop. 

Another  factor  to  discourage  the  workshop  participants  to  get  opportunities  for  skill  practice  after 
the  workshops  was  the  separation  between  the  workshop  classroom  and  the  computer  lab.  A while  after  the 
workshops,  its  participants  would  easily  forget  what  they  learned  if  they  did  not  apply  their  skills  to  a 
project  or  practice  the  skills.  Although  FACS  provided  a computer  lab  to  support  their  skill  application  or 
practice  after  the  workshops,  most  of  them  did  not  seriously  recognize  the  availability  of  the  service  and 
facility  for  the  support.  Since  the  workshop  was  not  held  in  the  computer  lab,  it  was  difficult  for  FACS  to 
make  them  familiar  with  the  resources  for  the  support.  Moreover,  for  those  who  preferred  self-learning  to 
one-on-one,  there  was  no  other  learning  assistance  available  than  class  handouts.  That  was  another  obstacle 
to  make  it  difficult  to  review  and  refine  their  skills  after  the  workshops. 
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Methods 


In  Spring  2006,  FACS  changed  the  workshop  classroom  and  format  to  meet  the  workshop 
participants’  expectations  and  enhance  their  learning  outcomes.  It  moved  the  workshop  classroom  from  a 
large-group  classroom  to  a renovated  small  computer  lab,  changed  the  workshop  format  from  lecture  to 
hands-on,  and  made  the  online  tutorials  available  to  them.  The  primary  goal  of  this  project  was  to  allow 
them  to  get  rich  learning  experiences.  Its  secondary  goal  was  to  let  them  recognize  its  support  service  and 
facility  available  to  them  to  encourage  them  to  use  the  resources. 

Workshop  classroom  design 

• FACS  had  a computer  lab  (Fig.  2)  with  3 PC  stations  for  supporting  faculty  projects  and  providing 
technology  consultation  and  one-on-one  training  sessions.  It  renovated  the  computer  lab  installing 
one  more  PC  station,  a LCD  projector,  and  a whiteboard  to  convert  the  lab  into  a web/multimedia 
training  classroom. 

• It  moved  its  workshop  classroom  from  a classroom  (Fig.  1)  with  20  PC  stations  for  large-group 
teaching  to  the  renovated  computer  lab  (Fig.  3)  with  4 PC  stations  for  small-group  hands-on 
training  classes.  While  the  multimedia  software  applications  were  installed  only  on  the  instructor’s 
station  in  the  large  classroom,  they  were  installed  on  every  station  in  the  lab. 

• It  installed  an  electronic  whiteboard  device,  Mimio  Xi,  to  make  the  class  teaching  more  interactive 
and  easy-to-follow.  It  allowed  the  workshop  participants  to  pay  attention  to  their  instructor  as  well 
as  the  instructor’s  activity. 

• It  placed  the  instructor’s  station  behind  the  participants’  stations  (Fig.  3)  so  that  he  can  easily  see 
their  learning  activities  and  watch  their  progress  through  their  stations’  monitors.  This  allowed 
him  to  confirm  that  every  participant  is  following  his  instruction  without  getting  lost  during  the 
class. 


Figure  1 The  classroom  for  large-group  lecture-format  classes 


Figure  2 The  computer  lab  before  the  renovation 


o 


Figure  3 The  renovated  computer  lab  for  small-group  hands-on  classes 
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Workshop  design 

• Converting  the  lecture-format  workshops  into  the  hands-on  workshops,  FACS  redesigned  the 
workshop  sessions  adding  more  numbers  of  examples  and  practice  materials  to  the  workshop 
instruction. 

• It  extended  the  workshop  time  from  50  minutes  to  60  or  80  minutes  to  allow  the  participants  to 
spend  more  time  for  practicing  skills. 

• The  instructor  stayed  at  the  classroom  to  answer  the  participants’  questions  on  the  skills  they 
learned  or  the  problems  in  their  projects  as  long  as  they  wanted  after  the  workshops  ended. 

• Converting  the  large-group  workshops  into  the  small-group  workshops,  it  limited  the  class  size  to 
4 participants  in  maximum.  It  offered  two  times  more  numbers  of  the  workshop  sessions  because 
of  the  small  class  size. 

• It  provided  online  tutorials  containing  screen  capture  videos  to  help  them  review  the  skills  they 
learned  at  any  time  they  wanted.  It  also  provided  downloadable  practice  fdes. 

Results 

FACS  implemented  the  workshop  improvement  project  in  Spring  2006  and  produced  the 
outcomes  of  the  successful  conversion  between  Spring  and  Fall  2006.  The  outcomes  were  drawn  from  the 
findings  from  the  participants’  feedback  on  the  workshop  evaluation  form  and  FACS  Center’s  faculty 
support  data.  The  evaluation  form  was  asked  to  be  submitted  at  the  end  of  the  workshops  and  the  support 
data  was  recorded  by  the  FACS  staff  at  the  time  of  the  service  provided.  FACS  initially  set  up  the  project’s 
target  outcomes  after  the  successful  conversion  as  below: 

Workshop  evaluation  feedback 

• Increase  of  the  number  of  the  workshop  participants  who  made  positive  feedback  on  the  two 
workshop  evaluation  questions  below: 

o Were  your  expectations  coming  into  this  session  met? 
o How  likely  is  it  that  you  will  use  something  that  you  learned  today? 

support  data 

Increase  of  the  number  of  faculty  members  asking  FACS  for  consultation  in  applying  technology 
to  their  teaching  or  research 

Increase  of  the  number  of  faculty  members  asking  FACS  for  one-on-one  training  at  the  FACS 
computer  lab 

Increase  of  the  number  of  faculty  members  using  the  FACS  computer  lab  for  their  projects 
Increase  of  the  number  of  faculty  members  asking  FACS  for  supporting  their  long-term  projects 
on  developing  their  course  materials 

The  participants’  feedback  from  the  workshop  evaluation  and  the  FACS  Center’s  faculty  support 
data  between  Spring  and  Fall  2006  were  compared  with  the  ones  between  Spring  and  Fall  2005,  when  the 
lecture-format  workshops  were  still  held  in  the  large-group  classroom. 

• The  percentage  of  the  workshop  participants  who  selected  “Exceeded”  on  the  question  about  the 
level  of  their  satisfaction  on  the  workshops  was  increased.  The  percentage  of  the  workshop 
participants  who  selected  “Very  likely  to  use”  on  the  question  about  the  usefulness  of  the  skills 
learned  in  the  workshops  was  also  increased. 

• There  was  an  increase  in  the  number  of  faculty  members  asking  consultation  and  one-one-one 
training.  There  was,  however,  no  increase  in  the  number  of  faculty  members’  requests  to  support 
their  projects.  Although  the  impact  of  the  revised  workshops  was  not  strong  enough  to  affect  the 
whole  faculty  community’s  acknowledgement  of  technology,  it  was  evident  that  the  workshops 
have  been  improved  regarding  the  aspect  of  hands-on  training. 

The  target  outcomes  that  FACS  initially  expected  to  get  from  the  project  were  intended  to  be 
interpreted  quantitatively.  After  the  projects  were  completed  in  two  semesters,  FACS  found  more  valuable 
issues  in  learning  attitudes  and  environments  than  in  the  increase  of  the  numbers.  Majority  of  the  findings 
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were  acquired  from  the  participants’  comments  in  the  workshop  evaluation  and  the  interactions  between  the 
participants  and  the  instructor  in  the  classes.  The  most  noticeable  responses  in  the  evaluation  were  their 
comments  directly  mentioning  the  “hands-on”  format  of  the  workshops.  This  clearly  reflected  how 
successfully  the  project’s  objectives  were  achieved. 

Findings 

The  small-group  (4  participants  in  maximum)  class  in  a small  classroom  (14  ft  x 15  ft)  supported 
the  hands-on  workshop  format.  It  allowed  the  participants  and  the  instructor  to  get  active  interactions  with 
each  other.  The  interactive  whiteboard  system,  Mimio,  installed  in  the  classroom  made  the  class  more 
active.  Since  it  allowed  the  instructor  to  control  his  station  on  a projected  screen,  the  participants  could  pay 
attention  to  him  as  well  as  his  instruction.  The  electronic  whiteboard  was  effective  in  the  classroom  which 
was  small  enough  to  let  the  instructor  and  the  participants  get  closely  together.  The  active  interaction 
helped  the  instructor  easily  recognize  each  participant’s  skill  level  and  learning  progress  during  the  class.  It 
also  helped  the  instructor  establish  personal  relationship  with  the  participants  even  after  the  workshops. 
More  participants  than  before  stayed  after  class  to  ask  questions,  discuss  about  their  own  projects  or 
arrange  consultation  sessions  or  one-on-one  sessions  as  follow-up.  This  played  an  excellent  role  to 
introduce  FACS  computer  lab  to  faculty  and  encourage  them  to  use  the  FACS  Center’s  support  service. 

Participants  preferred  building  up  their  own  skills  to  receiving  information  through  presentation  or 
demonstration  sessions  in  a class.  Learning  the  skills  by  practicing  following  their  instructor’s  instruction  in 
a class  allowed  them  to  realize  what  they  could  do  with  the  skills  for  themselves  as  well  as  how  the  skills 
actually  works.  This  was  the  mostly  mentioned  factor  to  make  them  feel  satisfaction  about  their  learning  in 
the  workshops.  Properly  selected  examples  and  well  designed  practice  files  were  the  key  to  the  successful 
practice  in  the  classroom  without  sacrificing  time  for  teaching.  Practicing  in  a class  also  helped  them  get 
their  confidence  on  their  ability  and  encourage  them  to  try  to  use  the  technology  and  to  take  more  advanced 
classes  or  other  related  classes.  For  those  who  wanted  to  review  or  practice  their  skills  even  after  the 
workshop,  FACS  created  online  tutorial  videos  and  made  them  available.  From  the  feedback  of  the 
workshop  participants  who  tried  the  tutorial  videos,  FACS  found  that  motion  screen  capture  videos  helped 
them  better  understand  instructions  and  more  easily  learn  skills  than  screen  capture  still  images  would. 

The  small-group  class  format  required  FACS  to  offer  two  times  more  number  of  classes  than  in 
the  previous  years.  Providing  more  variety  of  workshop  schedules  could  allow  faculty  to  avoid  schedule 
conflicts.  That  eventually  increased  their  chance  to  find  the  classes  they  could  attend,  while  this  required 
the  workshop  instructors  to  spend  more  time  to  teach  the  workshops  than  before.  In  addition  to  the  increase 
in  the  number  of  the  classes  offered,  the  expandable  class  time  was  another  key  element  for  the  successful 
workshop.  Even  after  the  class  ended,  the  instructor  expanded  class  time  for  answering  to  participants’ 
individual  questions  or  helping  their  skill  practices  as  long  as  they  wanted.  That  was  one  of  the  major 
elements  to  make  their  learning  experience  rich  and  eventually  make  them  satisfied  with  the  workshops. 

Issues 


The  small-group  class  format  required  more  number  of  classes  to  be  offered  and  more  staff  time 
for  class  preparation  and  teaching.  While  the  staff  time  had  to  be  increased  about  two  times  more  than  the 
one  in  the  previous  semesters,  the  number  of  the  participants  was  not  increased  that  much.  It  was,  however, 
noticeable  that  the  participants’  satisfaction  was  much  greater  than  the  one  in  previous  years.  It  is  said  that 
the  efficiency  in  the  class  productivity  was  not  much  high,  but  the  quality  of  the  classes  was  upgraded. 

Even  though  the  class  size  was  for  a small  group  of  participants,  there  was  still  a difficulty  to  deal 
with  participants  in  different  skill  levels.  The  hands-on  training  format  of  the  workshops  made  that 
difficulty  more  serious.  While  an  instructor  was  able  to  easily  recognize  their  skill  levels  in  that  classroom 
setup,  it  was  still  hard  to  keep  a balance  of  two  different  skill  level  groups  if  the  gap  between  the  two 
groups  was  too  big.  If  the  instructor  spent  too  much  time  for  taking  care  of  a participant  during  the  class, 
that  would  affect  the  class  time  for  all  other  participants.  For  the  efficient  class  management,  the  instructor 
had  to  ask  those  who  had  a difficulty  during  the  class  to  stay  after  the  class  or  arrange  a meeting  for  one-on- 
one  training  later. 

Most  of  the  classes  were  50  minutes  long.  Hoping  that  more  faculty  members  could  attend  the 
classes,  FACS  put  all  of  them  in  the  time  slot,  12:00  PM  - 12:50  PM.  The  class  time  was  good  enough  to 
attract  them  but  not  long  enough  for  hands-on  training.  The  classes  were  divided  into  two  or  three  short 
sessions  to  fit  the  classes  into  the  class  length.  This  actually  made  some  faculty  unable  to  complete  the 
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series  of  the  classes  on  a topic  because  of  their  schedule  conflicts.  Some  participants  suggested  that  the 
classes  could  be  two  hours  long,  1st  hour  for  instruction  and  2nd  hour  for  practice.  FACS  adopted  the 
suggestion  and  set  up  a plan  for  a pilot  study  to  investigate  the  new  hands-on  workshop  format  in  Spring 
2007.  In  the  instruction  session,  the  instructor  would  allow  participants  to  do  guided  practice  following  his 
step-by-step  instruction,  and,  in  the  practice  session,  he  could  provide  small  projects  to  let  them  practice  for 
themselves.  The  practice  session  could  be  optional  so  that  only  those  who  want  it  could  stay  after  the 
instruction  session. 

The  workshops  were  designed  and  organized  mainly  according  to  the  software  applications,  such 
as  PowerPoint,  PhotoShop,  Dreamweaver,  and  Flash.  This  workshop  design  would  restrict  them  to  a 
specific  software  application  and  make  them  lose  their  chance  to  see  other  options  or  alternatives  to 
accomplish  an  objective.  It  would  prevent  them  from  getting  a holistic  view  in  integrating  technology  into 
teaching.  Focusing  on  a specific  software  application  and  its  tools  would  be  an  effective  way  to  give  them  a 
solution  to  a specific  problem  but  not  to  encourage  them  to  get  more  fundamental  understanding  so  that 
they  can  apply  the  skills  to  other  projects  or  easily  understand  other  software  tools  and  find  more  creative 
ways  to  apply  the  tools  to  solve  their  problems.  Based  on  this  notion,  FACS  has  been  planning  another  pilot 
project  to  design  new  workshops  based  on  multimedia  tasks  rather  than  software  applications  and  to  add 
them  to  the  existing  workshops. 


Summary 

The  FACS  Center’s  effort  to  improve  the  instructional  technology  workshops  for  faculty 
development  by  converting  the  large-group  lecture-format  workshops  to  the  small-group  hands-on 
workshops  was  rewarded  with  more  learning  activities  created  in  the  workshop  classes  as  well  as  an 
increase  of  the  workshop  participants’  satisfaction  rate.  Instructor’s  step-by-step  instructions  for  guided 
practice,  the  increased  skill  practice  time,  the  classroom  setup  for  active  interactions  between  an  instructor 
and  participants,  and  the  online  tutorial  videos  were  the  key  elements  of  the  workshop  improvement  project 
to  help  the  participants  get  rich  learning  experiences.  While  more  staff  time  was  required  to  teach  and 
manage  more  number  of  workshops,  the  quality  of  the  workshops  was  greatly  improved.  Converting  the 
FACS  Center’s  computer  lab  to  the  multimedia  training  classroom  and  using  the  classroom  for  the 
workshops  made  the  FACS  Center  more  visible  to  the  faculty  so  that  the  faculty  members  could  easily 
recognize  its  support  services  and  facilities  available  to  them.  Raising  more  issues  on  designing  and 
managing  multimedia  training  workshops  for  further  projects,  this  project  led  the  FACS  Center  a step 
closer  to  the  creation  of  successful  technology  workshops  for  faculty.  The  workshop  and  its  classroom 
design  and  the  findings  of  this  project  will  help  academic  institutions  or  departments  who  either  plan  to 
provide  faculty  development  workshops  on  instructional  technology  or  who  wish  to  improve  their  current 
workshops  and  faculty  support  services. 
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Abstract 

New  computer-based  educational  materials  grounded  in  a constructivist  perspective 
of  learning  assume  fundamental  changes  in  classroom  instruction.  Designers  must 
therefore  support  teacher  learning  while  also  enabling  teachers  to  make  modifications  to 
meet  their  students  ’ needs  in  ways  that  maintain  fidelity  of  implementation.  The  study 
reported  here  examined  teachers  ’ assessment  goals  as  they  implemented  Alien  Rescue. 
Implications  for  the  design  of  educative  components  to  support  teacher  learning  and 
local  adaptation  of  constructivist  programs  are  discussed. 

Researchers  have  long  expressed  concern  about  the  modifications  teachers  make  to 
curricular  materials,  pointing  out  that  such  changes  have  the  potential  to  become  “lethal 
mutations”  (Brown  & Campione,  1996,  p.  291)  that  undermine  an  instructional  approach 
to  the  point  where  it  is  no  longer  effective.  Such  concerns  have  become  more  pressing  in 
recent  years  as  innovations  in  the  design  of  education  materials  have  drawn  not  just  on 
new  conceptions  of  learning  but  also  on  new  technologies.  In  order  to  support  the 
implementation  of  these  materials,  two  new  design  principles  have  been  advocated  in 
recent  years.  First,  researchers  have  argued  that  educational  materials  that  embody  new 
approaches  to  teaching  and  learning  must  support  teacher  learning  in  addition  to  student 
learning  (Ball  & Cohen,  1996;  Davis  & Krajcik,  2005).  Such  materials  are  referred  to  as 
educative,  and  are  designed  to  support  teachers  in  understanding  how  learning  is 
expected  to  occur  within  the  program  and  how  implementation  is  likely  to  affect  student 
outcomes.  Second,  given  that  teachers  are  responsible  for  fitting  curricular  materials  into 
the  contexts  in  which  they  work,  Barab  and  Luehman  (2003)  argue  that  designers  should 
consider  ways  in  which  to  allow  for  and  support  teachers’  “local  adaptations.”  This 
presents  a considerable  challenge,  in  that  designers  must  create  materials  that  are  flexible 
enough  to  be  effective  under  a range  of  conditions  while  structuring  support  so  that  the 
integrity  of  the  design  is  maintained. 

The  study  reported  here  examined  one  aspect  of  teacher  practice  - assessment  - for 
its  impact  on  the  implementation  of  a technology-based  program  informed  by  a 
constructivist  perspective  on  teaching  and  learning.  Research  suggests  that  the 
assessment  practices  commonly  used  in  K-12  environments  are  consistent  with  theories 
of  learning  that  dominated  education  in  the  early  part  of  the  20th  century,  but  are  at  odds 
with  the  new  conceptions  of  how  people  leam  that  have  emerged  from  the  constructivist 
perspective  (Shepard,  2000).  As  teachers  attempt  to  implement  constructivist  materials, 
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their  assessment  practices  are  likely  to  influence  their  decisions.  Designers  of  such 
materials  must  find  ways  to  support  teacher  learning  about  effective  assessment  practices 
that  are  consistent  with  the  theories  of  learning  underlying  these  materials  while 
simultaneously  supporting  teachers’  efforts  to  fit  novel  materials  into  their  curriculum. 
Such  an  objective  must  start  with  an  understanding  of  the  goals  and  beliefs  that  guides 
teachers’  decisions.  In  this  study,  we  examined  teachers’  implementation  of  Alien 
Rescue,  a 3-week  science  module,  focusing  on  the  research  question:  What  were 
teachers’  reasons  for  their  assessment  practices  while  implementing  Alien  Rescue? 

Methods 

Ten  6th  grade  science  teachers  were  interviewed  twice,  once  before  implementing 
Alien  Rescue  and  again  after  implementation.  Observations  were  also  conducted  in  five 
teachers’  classes,  and  the  results  were  used  to  formulate  additional  questions  for  the 
second  round  of  interviews.  The  interviews  and  field  notes  were  transcribed  and 
analyzed  using  the  constant  comparative  method  (Lincoln  & Guba,  1985),  including 
extensive  member  checking  to  support  the  trustworthiness  of  the  results. 

Results 

Seven  categories  emerged  from  the  data,  with  each  category  providing  a reason  for 
multiple  assessment  practices  used  by  teachers  during  their  implementation  of  Alien 
Rescue.  Each  of  the  following  reasons  underpinned  multiple  practices  and  were  offered 
by  a majority  of  the  teachers  in  this  study. 

Preparing  Students  for  Standardized  Tests 

Foremost  among  the  goals  that  drove  teachers’  assessment  decisions  were  pressures 
related  to  students’  performance  on  standardized  tests.  In  order  to  prepare  students  for 
statewide  standardized  tests,  the  district  in  which  this  study  was  conducted  had  recently 
implemented  a policy  of  semi-annual  benchmark  testing.  These  benchmarks  tests  were 
developed  by  the  district  to  reflect  the  types  of  items  on  the  new  state  exam  that  students 
take  in  5 th  and  10th  grades,  namely  multiple  choice  items,  some  of  which  examine  higher 
order  thinking.  Teachers  were  offered  training  on  the  development  of  multiple  choice 
tests  that  include  questions  representing  different  levels  of  learning,  including  higher- 
order,  problem-solving  items.  In  addition  to  the  benchmark  tests,  teachers  were  expected 
to  work  together  at  the  campus  level  to  develop  9-week  tests  to  administer  to  all  the 
students  at  their  school.  In  order  to  prepare  students  for  these  campus,  district,  and  state- 
level  tests,  teachers  felt  it  was  important  to  use  a multiple  choice  test  as  a summative 
assessment  of  every  unit,  even  for  PBL.  In  turn,  they  prepared  students  for  these  tests 
through  the  use  of  a variety  of  regular  tasks  that  required  students  to  respond  to  multiple 
choice  items,  such  as  the  warm-ups,  worksheets,  and  quizzes  teachers  used  in  Alien 
Rescue. 

Promoting  Productivity 

Also  at  the  forefront  of  factors  driving  teachers’  decisions  about  assessment  practices 
was  the  goal  of  maximizing  student  productivity.  Teachers  were  concerned  that  without 
direction,  students  would  either  engage  in  unproductive  tasks  or  become  frustrated  and 
disengage  from  the  program  entirely.  To  forestall  this,  teachers  applied  two  strategies. 
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First,  teachers  supplemented  Alien  Rescue  with  a variety  of  resources  that  helped  to 
structure  students’  work.  For  instance,  the  primary  reason  teachers  cited  for  giving 
students  paper-based  notebooks  was  that,  unlike  the  electronic  notebook  in  Alien  Rescue, 
the  paper-based  notebook  provided  headings  that  structured  the  problem  so  that  students 
did  not  waste  time  trying  to  figure  out  what  types  of  information  they  should  look  for. 
Second,  teachers  used  grading  practices  as  an  extrinsic  motivator  to  encourage  students  to 
stay  on-task.  For  example,  teachers  gave  grades  for  participation  to  discourage  off-task 
behavior.  Interestingly,  teachers  tended  to  argue  that  these  practices  were  necessary  only 
for  some  of  their  students.  For  example,  several  teachers  argued  that  note-taking  is  a 
difficult  skill,  one  that  could  be  overwhelming  for  children  of  average  and  below  average 
ability  level.  Similarly,  teachers  all  agreed  that  students  were  highly  motivated  during 
Alien  Rescue,  but  they  still  they  felt  that  extrinsic  motivators  were  necessary  “just  in 
case”  less  capable  or  interested  learners  were  unproductive. 

Adjusting  Class  and  Teacher  Activity 

Both  the  observations  and  interviews  provided  ample  evidence  that  teachers  used 
their  interactions  with  students  formatively,  that  is,  to  modify  both  whole  group 
discussions  and  their  own  activity.  Teachers  relied  less  on  graded  activities,  such  as 
notebooks,  worksheets,  and  quizzes,  to  make  these  decisions.  Teachers  saw  a disconnect 
between  the  ungraded  practices  they  used  fonnatively  to  assess  student  understanding 
and  the  practices  that  resulted  in  grades,  but  they  did  not  see  this  as  problematic.  In 
general,  teachers  saw  great  consistency  between  their  ungraded  assessments  of 
individuals  and  how  they  performed  on  graded  activities  and  products. 

Promoting  Excellence 

The  teachers  who  participated  in  this  study  all  argued  that  a major  goal  of  both  their 
teaching  and  assessment  was  to  help  students  produce  high  quality  work,  and  they  felt 
that  a strength  of  PBL  was  that  it  emphasizes  quality  over  quantity.  However,  though 
this  was  a goal  valued  by  all  teachers,  it  was  one  that  they  promoted  only  through  their 
ungraded  assessment  activities.  For  example,  teachers  used  their  infonnal  interactions 
with  students  to  encourage  deeper  level  thinking  and  better  quality  products,  but  none  of 
the  teachers  generated  any  grades  from  these  interactions.  In  fact,  some  of  the  comments 
teachers  made  suggest  that  they  believe  grading  would  be  counterproductive  to  their 
efforts  to  promote  excellence. 

Teacher  Accountability 

Teachers  cited  the  expectations  of  both  parents  and  administrators  as  impacting  their 
assessment  practices  in  three  ways.  First,  teachers  believed  that  both  administrators  and 
parents  expected  report  card  grades  to  be  based  on  numerous  grades  over  the  course  of 
the  6-week  marking  period.  Though  teachers  said  there  was  currently  no  requirement  in 
the  district  or  in  their  schools  for  a minimum  number  of  grades,  several  of  them  said  there 
had  been  such  requirements  in  the  past.  They  believed  that  if  a grade  were  challenged, 
they  would  need  to  show  that  it  was  based  on  a variety  of  products,  including  materials 
they  could  use  to  generate  “daily  grades.”  Of  course,  this  perceived  expectation  did  not 
determine  what  teachers  chose  to  use  to  generate  grades,  but  it  did  establish  a need  for 
something  that  could  be  graded.  Second,  teachers  felt  pressure  from  both  parents  and 
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administrators  to  show  that  grading  was  objective.  Teachers  also  cited  objectivity  as  a 
reason  to  justify  the  use  of  a multiple  choice  test  and  argued  that  one  of  the  advantages  of 
using  any  type  of  rubric  was  that  it  was  more  objective  than  assessing  a product 
holistically.  Most  teachers  felt  objectivity  was  important  in  grading,  equating  it  with 
“being  fair,”  but  some  felt  that  the  pressure  to  be  objective  negatively  influences  their 
teaching  and  ability  to  use  PBL  as  intended.  Finally,  teachers  felt  pressure  to  modify 
assessment  practices  so  that  learners  with  special  needs  would  receive  passing  grades. 
Teachers  adopted  practices  they  believed  were  needed  for  students  with  special  needs, 
then  applied  them  to  all  students.  In  this  case,  this  meant  that  grades  for  all  students  are 
likely  to  reflect  a combination  of  effort  and  achievement.  Teachers  felt  that  this  could  be 
problematic  in  that  grades  did  not  necessarily  reflect  students’  level  of  competence 
relevant  to  the  standards,  but  saw  it  as  unavoidable,  “just  a problem  with  grading  period. 
The  nature  of  the  beast.” 

Practicality 

Practical  considerations  guided  the  decisions  teachers  made  about  assessment.  Most 
notably,  concerns  about  the  time  needed  for  data  collection  and  grading  led  teachers  to 
use  certain  practices  and  avoid  others.  For  example,  a major  reason  offered  by  most 
teachers  for  using  multiple  choice  tests  was  that  they  were  easy  and  fast  to  grade. 
Likewise,  a common  reason  teachers  offered  for  using  quizzes  and  giving  participation 
grades  was  that,  as  one  teacher  explained,  “it’s  something  that’s  easy  to  grade  and  put  in 
the  grade  book.”  Ease  of  access  was  another  practical  consideration  for  teachers. 
Teachers  did  not  assess  either  students’  probes  or  electronic  notebooks  because  doing  so 
required  students  to  print  these  artifacts.  This  could  overwhelm  their  printer,  and  if 
students  forgot  to  print,  then  teachers  would  have  to  locate  these  artifacts  on  students’ 
hard  drives.  However,  for  the  most  part,  teachers  did  not  see  a need  to  assess  these 
artifacts.  They  believed  they  were  getting  the  infonnation  they  needed  about  students 
through  other  products  (such  as  worksheets,  wann-up  activities,  and  quizzes),  and  that 
these  other  products  were  more  easily  accessible. 

Consistency 

Creating  consistency  in  assessment  practices  across  the  school  year  was  at  least  a 
consideration  for  teachers.  For  example,  one  of  the  reasons  teachers  cited  for  continuing 
to  grade  participation  during  Alien  Rescue,  even  though  it  was  not  needed  for 
motivational  purposes  during  the  program,  was  that  it  was  something  they  used  during 
the  rest  of  the  year.  As  a second  example,  teachers  explained  that  one  reason  for  giving 
students  a multiple  choice  test  at  the  end  of  Alien  Rescue  was  that  students  expected  it 
because  they  had  such  tests  at  the  end  of  other  units.  Goals  other  than  consistency 
provided  the  primary  reasons  for  teachers’  choice  of  both  of  these  practices,  and  several 
teachers  commented  that  consistency  in  assessment  practices  was  not  a driving  factor  in 
their  assessment  decisions,  but  they  believed  that  students  were  more  comfortable  and 
performed  better  when  assessment  practices  were  familiar  and  used  throughout  the  year. 

Discussion 

Based  on  the  results  of  this  study,  we  suggest  four  design  guidelines.  The  first  two 
aim  to  support  teachers  in  modifying  a PBL  program  to  address  what  they  perceive  as  the 
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needs  of  their  own  students  within  their  local  class  and  school  context.  The  second  two 
address  educative  goals  for  programs  like  Alien  Rescue. 

Embed  multiple-choice  questions  within  PBL  activities 

We  argue  for  this  course  of  action  with  trepidation  because  it  is  a prime  example  of  a 
practice  that  is  inconsistent  with  the  social  constructivist  paradigm  in  which  PBL  is 
situated.  However,  given  the  pressures  created  by  high  stakes  testing,  it  is  the  adaptation 
most  teachers  are  likely  to  make  themselves  if  it  is  not  provided  within  the  program. 

PBL  programs  can  offer  teachers  control  over  number  of  items  presented,  topics 
addressed,  and  level  of  difficulty,  and  even  let  them  individualize  these  decisions  based 
on  their  perceptions  about  students’  needs.  Having  the  items  developed  by  designers 
offers  several  advantages.  First,  the  program  can  retain  control  of  timing  of  the 
presentation  of  items,  for  instance,  presenting  a given  multiple  choice  item  only  after 
students  have  accessed  the  section  of  the  program  that  presents  information  on  it.  In  this 
way,  these  items  would  follow,  not  drive,  student  activity.  Second,  designers  can  involve 
assessment  experts  who  are  more  likely  to  develop  valid  objective  items  capable  of 
encouraging  higher-order  thinking.  This  can  relieve  teachers  of  the  responsibility  for 
developing  these  items,  a task  that  teachers  in  this  study  found  difficult.  Well-chosen 
items  may  actually  help  students  to  recognize  topics  pertinent  to  solution  development 
rather  than  focusing  them  on  the  acquisition  of  discrete,  decontextualized  facts.  Finally, 
the  data  gathered  on  student  performance  on  these  items  can  be  presented  to  teachers  by 
the  program  along  with  suggestions  on  how  to  address  misconceptions  and  learning 
needs. 

Provide  modifications  and  scaffolds  within  the  program  that  teachers  can  assign  to 
individual  students  based  on  special  needs 

In  this  study,  teachers  made  modifications  to  the  program  for  all  students  that  were 
based  on  the  perceived  needs  of  only  a few.  Given  teachers’  concerns  that  PBL  is  too 
challenging  for  some  students,  allowing  teachers  to  provide  greater  structure  and  support 
for  some  may  make  it  unnecessary  for  teachers  to  modify  the  program  for  all  students. 

The  program  can  even  be  designed  to  provide  information  to  teachers  about  which 
students  appear  to  be  floundering.  For  instance,  in  Alien  Rescue,  data  on  the  notes 
recorded,  tools  accessed,  and  probes  launched  can  allow  teachers  to  assess  student 
productivity  quickly.  By  comparing  learners’  patterns  to  those  of  students  who  have  been 
successful  with  a program  in  the  past,  it  may  be  possible  for  the  computer  to  alert 
teachers  to  which  students  may  need  support,  allowing  teachers  to  target  individual 
students  for  help  rather  than  providing  unnecessary  extrinsic  motivators  and  structure  for 
the  entire  class. 

Present  research  findings  on  motivation  and  teaching  practices,  especially  the 
impact  of  grades  on  intrinsic  motivation 

The  use  of  extrinsic  motivators,  particularly  grades,  is  so  entrenched  in  K-12  culture 
that  a combination  of  convincing  arguments,  research-based  evidence,  and  alternative 
practices  is  necessary  to  help  teachers  consider  how  new  approaches  such  as  PBL  interact 
with  student  motivation.  Educative  components  can  present  explanations  of  PBL  and  its 
effect  on  students’  motivation  as  well  as  research  findings  on  the  impact  of  extrinsic 
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motivators  on  intrinsic  motivation.  The  assessment  tool  itself  can  avoid  supporting 
practices  that  may  undermine  intrinsic  motivation  while  supporting  practices  that  focus 
on  enhancing  the  quality  of  students’  work.  Developing  convincing  arguments  capable 
of  supporting  teacher  change  will  be  an  important  focus  of  our  future  design  and  research 
efforts. 

Emphasize  the  value  of  student  reflection  on  their  process  and  solution  through  self 
and  peer  assessment 

Engaging  students  in  reflection  on  the  tasks  they  perform  and  in  assessment  of  the 
products  they  develop  may  enhance  their  problem-solving  process  and  their  standards  for 
quality  work.  Given  teachers’  reluctance  to  dedicate  time  to  engage  students  in  formative 
assessment,  we  plan  to  build  scaffolds  for  it  directly  into  Alien  Rescue.  The  goal  here 
would  be  to  make  formative  assessment  a seamless  component  of  students’  process  and 
relieve  teachers  of  at  least  some  of  the  responsibility  for  guiding  it.  At  the  same  time,  we 
plan  to  develop  educative  components  that  present  research-based  knowledge  about 
engaging  students  in  formative  assessment  (Black  & Wiliam,  1998)  that  help  teachers  to 
understand  its  potential  benefits  and  how  to  facilitate  it  effectively. 
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The  purpose  of  this  research  project  was  to  open  discussions  among  faculty  and  graduate  students  about  the 
importance  of  mentoring  graduate  students  to  teach  in  distance  education,  specifically  online  environments,  and 
what  that  mentoring  should  look  like  in  terms  of  content  and  relationship.  The  purpose  of  this  research  is  twofold. 
First,  there  is  a need  to  open  discussions  among  faculty  and  graduate  students  about  the  importance  of  mentoring 
graduate  students  to  teach  in  online  learning  environments  through  modeling  behaviors  and  mentoring  while  they 
are  fully  involved  in  the  teaching  of  an  online  course.  A subsequent  purpose  was  to  develop  guidelines  that  other 
faculty  and  graduate  students  can  utilize  as  they  engage  in  the  mentoring  relationship  which  focuses  on  the  diffusion 
of  best  practices  for  online  learning  The  research  questions  included  (1)  What  does  the  mentoring  process  look  like, 
(2)  What  are  the  major  processes  and  pedagogies  related  to  teaching  in  an  online  environment  that  graduate  students 
need  to  be  mentored  about,  and  (3)  How  do  graduate  students  perceive  the  notion  of  transferring  their  experiences  as 
a student  in  an  online  course  environment  to  teaching  in  an  online  course  environment?  The  results  of  this  study, 
including  emerging  themes,  offer  implications  for  the  mentoring  relationship  as  well  as  the  development  of 
guidelines  for  mentoring  graduate  students  in  best  practices  for  online  learning. 

Background 

In  education,  a critical  piece  of  primary  and  secondary  school  teachers’  training  is  being  mentored  during 
their  student  teaching  by  someone  with  experience  to  give  feedback  (Ingvarson  & Kleinhenz,  2003),  support 
(Cameron- Jones  & O'Hara,  1997),  help,  and  orient  them  to  the  struggles  of  a new  teacher  (Schreiver,  1999).  In 
academia,  the  mentoring  relationship  can  help  graduate  students  make  the  transition  from  undergraduate  to  doctoral 
work  and  prepare  them  for  their  future  careers  (Austin,  2002).  For  the  protege,  a mentoring  relationship  might 
provide  networking  opportunities  and  important  contacts,  guidance  on  coursework  and  opportunities,  experience, 
and  safety  (often  political)  through  the  protege’s  graduate  career.  A longitudinal  study  of  individuals  working  in 
post-graduate  careers  found  that  there  were  significant,  lasting  differences  between  students  who  were  proteges  and 
those  who  were  not  in  areas  such  as  career  planning  and  socialization  (Chao,  1997).  For  the  mentor,  the  benefits  of 
the  mentoring  relationship  may  include  work  towards  service  requirements,  and  rekindling  excitement  about 
teaching  (Gaia,  Corts,  Tatum,  & Allen,  2003). 

Research  on  mentoring,  particularly  at  the  graduate  level,  is  limited.  Johnson  and  Huwe  (2003)  point  out 
“most  graduate  programs  do  not  collect  data  regarding  mentoring  of  their  own  students”  (p.  9)  because  it’s  difficult 
for  prospective  students  in  a program  to  evaluate  it  based  on  mentoring,  and  mentoring  varies  across  disciplines, 
making  assessment  and  comparison  complex.  Some  data  on  mentoring  in  academia  comes  from  the  field  of 
psychology  (Johnson  & Huwe,  2003).  One  study  found  that  of  eight  hundred  responses  from  psychologists  around 
the  United  States,  two  thirds  had  a mentor,  and  ninety-one  percent  of  those  mentored  viewed  it  positively  (Clark, 
Harden,  & Johnson,  2000). 

From  the  literature  on  mentoring,  which  has  a strong  emphasis  on  mentoring  pre-service  teachers,  we  know 
that  mentors’  roles  can  vary  greatly.  A popular  mentoring  model  divides  a mentor’s  responsibilities  into  two  parts, 
career  functions  and  psychosocial  functions  (Kram,  1983).  Career  functions  include  sponsorship,  exposure, 
coaching,  protection,  and  challenging  assignments.  Psychosocial  functions  are  role  modeling,  acceptance  and 
confirmation,  counseling,  and  friendship.  Modeling  might  include  asking  a protege  to  observe  a specific  professional 
task  like  teaching  or  presenting,  and  helping  them  practice  it  themselves  (Johnson,  2002).  A more  specific  mentoring 
model,  proposed  by  Anderson  and  Shannon  (1988)  is  based  on  historical  and  current  mentoring  models.  Their  model 
includes  five  attributes:  the  process  of  nurturing,  the  act  of  serving  as  a role  model,  the  five  mentoring  functions 
(teaching,  sponsoring,  encouraging,  counseling,  and  befriending),  the  focus  on  professional  and  personal 
development,  and  an  ongoing  caring  relationship  (Anderson  & Shannon,  1988,  p.  40). 

Another  important  distinction  for  this  study,  as  defined  by  Burlew  (1991),  is  the  distinction  between 
primary  and  secondary  mentoring  relationships.  A primary  mentoring  relationship  is  long-term,  lasting  the  length  of 
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the  protege’s  experience  at  the  school  and  beyond.  Secondary  mentoring  relationships  are  shorter,  usually  relating  to 
a specific  task.  The  current  study  will  describe  a specific  secondary  mentoring  relationship  in  which  the  proteges 
were  coached  in  how  to  teach  an  online  course  under  the  tutelage  of  an  experienced  online  teacher  who  acted  as  a 
mentor. 

Learning  to  Teach,  Learning  to  Teach  Online 

It’s  common  for  instructors  who  have  never  taken  or  taught  an  online  course  to  be  asked  to  convert  an 
existing  course  to  an  online  format,  or  to  teach  an  existing  online  course.  In  fact,  it  is  not  uncommon  for  a graduate 
with  little  to  no  teaching  experience  to  be  asked  to  teach  a class  within  their  discipline  (Darling  & Earhart,  1990; 
“Some  universities  shift  future  professors'  focus  to  teaching”,  2006).  The  assumption  about  teaching  has 
traditionally  been  that  if  you  possess  the  content  knowledge  then  you  can  teach  the  content  (Darling  & Earhart, 
1990);  similarly,  an  assumption  is  made  that  teaching  online  and  face-to-face  are  basically  the  same,  though 
instructors  may  be  asked  to  learn  new  technology.  However,  technology  is  only  one  of  the  differences  between  the 
two  classroom  formats.  “For  academics  to  successfully  make  the  transition  to  become  online  teachers  or  learning 
facilitators,  they  must  do  more  than  develop  new  technological  skills.  Online  development  and  delivery  requires  new 
pedagogical  approaches,  challenging  previous  practices  with  regards  to  assessment,  group  interaction  and 
student/teacher  dialogue”  (Ellis  & Phelps,  2000,  p.  27).  Online  instructors  also  have  different  roles,  or  perhaps  more 
precisely,  different  strategies,  approaches,  and  emphases  than  instructors  in  traditional  classrooms  (Goodyear, 
Salmon,  Spector,  Steeples,  & Tickner,  2001;  Kulski,  Boase-Jelinek,  Pedalina,  Quinton,  O'Connor,  2002),  with  the 
general  categories  identified  as:  pedagogical,  managerial,  social,  and  technical  (Berge,  1995).  However,  online 
instructors  are  challenged  in  playing  the  above  roles,  especially  the  pedagogical  and  social  roles  (Shrivastava,  1999). 
Moreover,  research  shows  that  effective  online  learning  depends  on  how  online  instructors  develop,  design,  and 
facilitate  teaching  and  learning  activities,  including  the  provision  of  support  for  learning  activities  (Berge,  1995;  Liu, 
Bonk,  Magjuka,  Lee  & Su,  2005).  Given  these  distinctions  and  research  findings,  it  is  presumed  that  the  task  of 
developing  and  teaching  an  online  course  would  require  different  and/or  additional  training  (Barker,  2002,  Bennett 
& Marsh,  2002). 

A recent  article  from  CNN.com  (“Some  universities  shift  future  professors'  focus  to  teaching”,  2006) 
discusses  a shift  to  teaching  from  research  as  a focus  at  some  universities  as  the  scarcity  of  academic  jobs  continues. 
Accordingly,  some  universities  are  taking  teacher-training  more  seriously  to  not  only  enhance  their  own 
undergraduates’  education  but  also  to  increase  their  graduate  students’  chances  of  employment  in  academia.  The 
article  also  describes  how  some  universities,  including  Washington,  Colorado  and  Michigan  have  developed  centers 
that  train  and  mentor  teaching  assistants.  However,  according  to  a recent  survey,  only  about  one-half  of  4,000 
doctoral  students  from  27  universities  required  students  to  serve  as  teaching  assistants.  A doctoral  candidate  who 
was  interviewed  for  the  article  had  the  following  to  say  in  terms  of  the  necessity  of  being  able  to  teach,  “How  do  you 
handle  a racial  comment  that  targets  another  student?  How  do  you  manage  classroom  tension?”. (“Some  universities 
shift  future  professors'  focus  to  teaching”,  2006).  The  researchers  believe  that  this  is  compounded  with  a lack  of 
online  teaching  experience  and  knowledge,  as  the  response  to  these  questions  would  differ  depending  upon  course 
format,  traditional  or  online.  If  Berliner’s  (1986)  pedagogical  model  is  the  basis  for  training  graduate  assistants,  and 
includes  two  domains:  (1)  knowledge  as  subject  matter  and  (2)  knowledge  of  organization  and  management  of 
classrooms  (in  Darling  and  Earhart,  1990),  then  the  researchers  propose  that  there  should  also  be  consideration  for 
an  additional  layer — course  format. 

In  response  to  this  challenge,  colleges  and  universities  have  developed  curriculums  and  programs  to 
educate  instructors  on  how  to  teach  online  (Salter  & Hansen,  1999,  Prendergast,  2004).  For  example,  some  graduate 
programs  encourage  students  and  instructors  to  attend  Brown  Bags  and  other  informal  sessions  to  learn  more  about 
teaching  online,  while  another  program  promotes  a team  of  mentors  to  aid  with  technological  concerns  (Gray  & 
McNaught,  2001).  A common  online  professional  development  approach  to  train  faculty  to  teach  online  is  “some 
combination  of  workshops,  tutorials,  and  technical  support”,  but  this  approach  fails  to  integrate  pedagogy  and 
technology  and  ignores  the  relationship  between  the  two  (Koehler  & Mishra,  2004,  p.  8).  In  fact,  the  literature 
reveals  a gap  in  the  training  of  instructors  in  pedagogical  approaches  and  limited  opportunities  for  them  once  they 
arrive  in  academia  (Darling  & Earhart,  1990;  Kember  & McKay,  1996,  in  Louie,  Drevdahl,  Purdy  & Stackman, 
2003).  As  such,  we  now  must  consider  the  additional  obstacle  of  a lack  of  teaching  experience  in  online  courses  and 
how  it  impacts  our  graduate  students  as  they  enter  the  market.  Given  this,  it  is  the  responsibility  of  graduate 
programs  to  allow  graduate  students  the  experience  of  teaching  online  with  the  assistance  of  a mentor  who  is  an 
expert  in  best  practices  for  online  courses  for  the  following  reasons  (1)  with  the  ever-growing  number  of  online 
courses  a new  instructor  may  find  that  they  are  expected  to  teach  online,  (2)  because  it  is  different  from  traditional 


371 


teaching  in  terms  of  pedagogical,  managerial,  social,  and  technical  aspects,  and  (3)  to  be  able  to  offer  graduate 
students  not  only  the  confidence  but  the  marketability  of  having  this  experience. 

Methods 

This  research  study  stems  from  a constructivist  paradigm  (Patton,  2002),  specifically  a socio-cognitive 
theoretical  framework  (Bandura,  1977),  and  employs  a collaborative  self-study  methodology  (Louie,  Drevdahl, 
Purdy  & Stackman,  2003)  to  study  the  mentoring  process  of  two  graduate  students  by  a faculty  member  in  the 
design,  development  and  instruction  of  an  online  course.  The  basis  of  the  self-study  was  a discussion  between  the 
mentor,  Jennifer  and  the  first  graduate  protege,  Hans.  Working  together  in  the  Fall  of  2004  on  a research  project 
related  to  social  aspects  of  online  learning,  a discussion  arose  that  pertained  to  Hans'  plans  to  work  in  the  academy 
upon  graduation  and  the  types  of  experiences  that  would  prove  beneficial  to  him  as  a graduate  student.  We  decided 
that  Hans  should  have  some  experience  with  teaching  in  an  online  environment  as  it  varies  considerably  from  the 
traditional  classroom.  At  that  point,  fall  2004,  we  began  meeting  occasionally  to  discuss  what  this  experience  would 
look  like  in  order  to  provide  Hans  with  a meaningful  experience.  After  several  discussions  we  decided  that  Hans 
would  not  serve  as  a teaching  assistant,  but  rather  a co-instructor,  providing  him  with  the  possibility  of  having  more 
authority  and  input;  we  decided  we  would  devise  the  experience  as  the  course  progressed.  The  second  mentoring 
opportunity  arose  following  my  (Jennifer)  satisfaction  and  reflectiveness  with  the  first  experience  combined  with 
Ann's  interest  in  obtaining  experience  in  teaching  this  particular  course  online.  The  self-study  evolved  from  the 
process  by  which  we  engaged  in,  the  process  of  determining  the  areas  where  proteges  felt  they  needed  guidance. 

The  collaborative  nature  of  this  approach  not  only  allowed  the  graduate  students/proteges  to  develop  their 
online  teaching  and  development  skills  but  also  allowed  the  faculty  member/mentor  to  reflect  on  her  online  teaching 
beliefs  and  practices.  The  literature  on  collaborative  self-study  discusses  the  benefits  of  this  type  of  research 
including  social  support,  a culture  of  reflectiveness  resulting  in  higher  level  discourse,  and  increase  chances  that 
faculty  will  create  more  generalizable  knowledge  (Louie,  Drevdahl,  Purdy  & Stackman,  2003).  It  is  the  final  benefit 
that  is  often  overlooked  by  self-study  research  but  is  a major  focus  of  our  research  (Louie,  Drevdahl,  Purdy  & 
Stackman,  2003;  Clandinin  & Connelly,  2000). 

This  particular  study  utilized  a process  model  of  collaborative  self-study  which  emphasizes  three  main 
phases:  (1)  the  assessment  phase,  where  researchers  evaluate  self-readiness,  access  to  collaborators,  and  defining  a 
focus  with  academic  discourse,  (2)  the  implementation  phase,  where  researchers  select  data  collection  and  analysis 
methods  while  assuring  validity  ; and  (3)  the  dissemination  phase,  where  researchers  share  not  only  their  results  but 
implications  for  a more  generalizable  audience  (Louie,  Drevdahl,  Purdy  & Stackman,  2003).  It  is  the  final  phase  that 
is  often  overlooked  by  self-study  research  but  is  a major  focus  of  our  research  (Clandinin  & Connelly,  2000;  Louie, 
Drevdahl,  Purdy  & Stackman,  2003). 

The  validity  for  this  research  lies  in  the  dependability  of  our  study  design  (ongoing  field  texts  consisting  of 
narratives,  e-mails,  and  conversations);  the  authenticity  of  our  study  design  (the  reflection  of  our  own  viewpoints 
and  the  evolution  of  practices  and  beliefs  based  not  only  our  own  reflections  but  the  reflections  of  our  team 
members),  and  of  course,  the  triangulation  of  the  multiple  sources  which  we  refer  to  as  field  texts  (Patton,  2002,  pg. 
546).  Validity  of  socially  constructed  research,  such  as  this  study,  is  embedded  in  the  credibility,  transferability, 
dependability,  and  confirmability  of  a study  (Lincoln  and  Guba,  1986,  in  Patton,  2002,  pg.  546.). 

Role  of  the  Researchers 

Three  researchers  participated  in  this  study:  the  faculty  member/mentor  and  two  graduate  students/proteges. 
The  faculty  member/mentor  has  over  6 years  experience  designing,  developing,  and  teaching  in  online  environments 
and  several  additional  years  with  experience  as  a student  in  online  courses.  The  first  graduate  student/protege,  Hans, 
was  mentored  during  the  spring  2005  semester  during  the  course.  Hans  had  taken  two  online  courses  prior  to 
teaching  the  course  involved  in  this  research  (from  instructors  other  than  the  mentor  for  this  study).  He  had 
previously  taken  the  course  serving  as  the  focus  here,  offered  in  a traditional  format  by  the  mentor  of  this  study.  In 
addition,  Hans  received  independent  study  credits  for  his  role  in  the  co-teaching  of  the  course  in  the  spring  of  2005. 
The  second  graduate  student/protege,  Ann,  was  mentored  during  the  spring  2006  semester  for  the  same  course  as 
Hans.  Ann  had  previously  taken  one  online  course  prior  to  co-teaching  the  course  that  serves  as  the  focus  here, 
which  happened  to  be  the  same  online  course  taken  with  the  mentor  of  the  study.  As  a protege,  Ann  helped  co- 
instruct as  partial  fulfillment  of  doctoral  fellowship  requirements. 

The  mentor  and  proteges  met  weekly  over  the  course  of  the  semesters  of  their  involvement  with  the  course 
as  well  as  for  additional  sessions  following  the  semester  to  discuss  course  design,  facilitation,  assessment,  students 
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learning,  content,  and  teaching  experiences  among  other  things.  The  researchers  also  frequently  engaged  in 
discussions  via  e-mail  throughout  each  week  related  to  the  practices  and  processes  related  to  the  course.  Finally,  the 
researchers  met  throughout  the  research  process,  with  Ann  becoming  part  of  the  project  during  her  semester  of 
involvement,  spring  2006. 

Research  Context 

The  context  of  the  study  that  served  as  a basis  for  the  mentoring  relationship  was  an  online,  graduate  level 
course.  Integration  and  Management  of  Computers  in  Education,  and  occurred  during  the  spring  2005  and  the  spring 
2006  semesters.  At  the  beginning  of  the  course,  there  was  a single  face-to-face  meeting  that  served  as  an  orientation 
to  the  course  and  the  online  format.  Students  were  expected  to  complete  several  required  course  projects  and 
activities  on  an  individual  basis  including  (1)  an  Instructional  Integrated  Project,  (2)  a Case  Study  (e.g.  apply  a 
solution  to  a practical,  realistic  problem  that  may  be  encountered  in  the  application  of  technology  into  the  classroom 
or  business  environment.),  and  (3)  an  Exemplary  Technology  Educator  Vision  (pre  and  post  course).  Students  were 
also  encouraged,  but  not  required,  to  work  with  a peer  for  the  Issues  Project  (which  was  based  on  a topic  of  the 
students'  choice  and  included  a literature  review  and  an  online  discussion  lead  during  the  course). 

The  course  participants  in  the  online  course  for  the  spring  2005  version  of  the  course  included  15  graduate 
students.  Eight  of  the  participants  were  administrators  (e.g.  technology  directors,  curriculum  directors,  principals, 
assistant  principals)  all  of  which  were  also  former  teachers;  an  additional  four  participants  were  also  current  or 
former  teachers.  Seven  of  the  doctoral  students  belonged  to  a cohort  program  in  educational  administration. 

Fourteen  of  the  students  were  pursuing  a graduate  degree  (5  at  the  masters’  and  nine  at  the  doctoral  level);  one 
student  was  a post-baccalaureate.  Ten  participants  were  female  and  five  were  male.  Of  the  fifteen  students,  two  were 
international  students.  Student  evaluations  for  this  course  indicate  that  students  thought  the  course  and  the 
instructors  performed  well.  Moreover,  two  important  indicators  of  good  teaching  practices  online,  facilitation  and 
feedback,  were  measured  by  questions  indicated  in  the  table  1 and  indicate  that  students  rated  them  on  average  as 
very  good. 


Table  1.  Course  Evaluations,  Spring  2005 


Responses  to  Course  Evaluation  (n=12) 

*Ratings  based  on  scale  of  Excellent  (5)  to  Very  Poor  (1) 

Overall  I would  rate  this  course  as: 

4.25 

Overall  I would  rate  these  instructors  as: 

4.58 

The  amount  of  facilitation  provided  by  the  instructors  was  satisfactory. 

4.17 

I received  prompt  feedback  from  the  instructors  when  I had  questions  or 
needed  clarification. 

4.5 

12 

Adequate  feedback  is  provided  to  guide  my  progress  in  this  course  (e.g. 
assignments,  discussions) 

4.0 

12 

The  course  participants  in  the  online  course  for  the  spring  2006  version  of  the  course  included  18  graduate 
students.  Eleven  of  the  participants  were  administrators  (e.g.,  curriculum  directors,  principals,  assistant  principals) 
all  of  which  were  also  former  teachers  and  belonged  to  a cohort  program  in  educational  administration;  an  additional 
two  participants  were  also  current  teachers.  Seventeen  of  the  students  were  pursuing  a graduate  degree  (four  at  the 
masters’  and  thirteen  at  the  doctoral  level);  one  student  was  a post-baccalaureate.  Ten  participants  were  female  and 
eight  were  male.  Of  the  fifteen  students,  two  were  international  students.  Student  evaluations  for  this  course  also 
indicate  that  students  thought  the  course  and  the  instructors  performed  well  (see  table  2.).  Similar  to  the  spring  2005 
course  evaluations,  students  for  spring  2006  also  indicated  a high  degree  of  satisfaction  in  regards  to  facilitation  and 
feedback  from  instructors. 


Table  2.  Course  Evaluations,  Spring  2006 


Responses  to  Course  Evaluation  (n=13) 

*Ratings  based  on  scale  of  Excellent  (5)  to  Very  Poor  (1) 

Overall  I would  rate  this  course  as 

Overall  I would  rate  these  instructors  as 

4.33 

The  amount  of  facilitation  provided  by  the  instructors  was  satisfactory. 

I received  prompt  feedback  from  the  instructors  when  I had  questions  or 

4.75 
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needed  clarification. 

Adequate  feedback  is  provided  to  guide  my  progress  in  this  course  (e.g. 
assignments,  discussions) 

4.10 

The  sixteen  week  course  was  broken  up  into  learning  modules,  each  covering  several  weeks,  and  defined 
by  a particular  overarching  topic:  why  integrate  technology,  how  to  integrate  technology,  distance  learning, 
management  of  technology,  and  future  trends  and  issues.  Each  week  students  were  required  to  respond  to  one  or 
two  discussion  questions  (threaded  discussions),  with  the  timeframe  of  a week,  comprising  the  initial  student 
response  as  well  as  required  responses  to  peers. 

Data 


The  data  for  this  study  was  collected  over  an  18-month  period,  and  culminated  in  what  the  researchers  refer 
to  as  field  texts.  Field  texts,  as  defined  by  Clandinin  & Connelly  (2000),  are  different  forms  of  texts  that  can  include 
autobiographical  writing,  journals,  field  notes,  etc.  that  are  interwoven  throughout  the  whole  study  (pg.  116).  For  our 
purposes  field  texts  included  mentor  observation  notes,  informal  journal  reflections  by  the  proteges,  conversations 
(face-to-face  and  via  e-mail),  mentor's  journal,  and  proteges  reflective  papers  following  their  involvement  with  the 
course.  The  purpose  of  the  final  reflective  papers  from  the  proteges  was  to  allow  them  to  reflect  on  themselves  as 
educators,  in  terms  shifting  teacher  beliefs,  and  as  professionals  involved  in  distance  education.  Journals  kept  by 
teachers  are  considered  a valuable  data  source  as  it  "acts  a narrative  technique  and  records  events,  thoughts,  and 
feelings  that  have  importance  for  the  writer”  (Anderson,  Herr  & Nihlen  as  cited  in  Mills,  2003  p.  67).  The  data  was 
analyzed  using  the  constant  comparative  method  (Glaser  & Strauss). 

Field  Texts 

The  field  texts  were  kept  by  the  researchers  throughout  the  span  of  the  course  and  were  comprised  of  notes 
that  detailed  topics  and  questions  the  mentor  and  proteges  discussed  at  weekly  meetings.  The  field  texts  also 
included  semi-structured  narratives  written  by  the  mentor  and  proteges  that  examined  issues  related  to  assumptions 
and  practices  that  were  utilized  within  the  course.  Finally,  the  field  texts  included  the  ongoing  electronic  dialogue 
between  the  researchers  as  the  course  progressed. 

More  specifically,  the  field  texts  looked  to  expectations  of  what  the  mentor  would  focus  on  modeling  for 
the  proteges,  teaching  beliefs  and  assumptions  on  the  part  of  both  researchers,  and  unexpected  outcomes  that  came 
about  as  the  course  and  the  mentoring  process  unfolded.  For  example,  situations  related  to  experiences  that  the 
proteges  had  not  had  the  opportunity  to  consider  as  a possible  area  for  mentoring.  Specifically,  the  proteges  were 
responding  to  the  following  overarching  topics  in  his  field  texts  related  to  teaching  in  an  online  environment: 
assumptions  about  teaching  online,  what  they  were  looking  for  as  proteges,  what  they  received  mentoring  about, 
course  design  (assumptions  and  observations),  implementation  of  the  course  activities  (assumptions  and 
observations),  and  class  management  (assumptions  and  observations).  The  mentor  was  responding  to  her  beliefs  and 
assumptions  about  online  learning,  specifically  her  previous  experience  with  online  learning  (as  student  and 
instructor),  how  she  learned/adapted  to  new  environment  (as  student  and  instructor),  attitude  and  perceptions  toward 
online  learning  (as  student  and  instructor),  what  she  had  planned  going  into  the  experience  to  mentor  the  proteges 
about,  what  she  felt  she  had  mentored  the  proteges  about,  what  she  felt  the  proteges  challenged  her  about  in  terms  of 
beliefs  and  assumptions,  how  she  went  about  developing  and  designing  a course  and  why,  her  perception  of  the 
instructor's  role  in  discussions  and  the  course  in  general,  and  her  experience  mentoring  graduate  students  in  this 
area. 

Limitations 

This  research  is  based  on  the  personal  experiences,  teaching  and  instructional  beliefs  of  the  researchers,  all 
of  whom  reside  in  an  Educational  Technology  program  situated  within  a College  of  Education,  and  therefore  results 
may  differ  from  other  disciplines.  An  additional  limitation  to  the  research  results  from  the  fact  that  the  proteges  were 
not  fully  involved  in  the  course  development  as  typical  course  constraints  existed,  including  previous  versions  of  the 
course,  it  had  to  meet  program  requirements,  and  it  needed  to  align  with  the  appropriate  ISTE  standards.  However, 
the  topics  and  discussion  questions  allowed  for  some  flexibility  and  the  proteges  were  involved  in  development  of 
those  areas.  Moreover,  the  course  design  was  discussed  between  mentor  and  proteges,  with  the  mentor  stressing  that 
while  her  practices  stem  from  experience  and  research  findings  about  best  practices  for  online  learning,  there  are 
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also  many  other  factors  that  go  into  course  development,  such  as  personal  instructional  strategies,  instructors' 
experiences  and  observations,  and  course  content. 


Initial  Results 

The  Mentoring  Process:  Establishing  the  Power  Relationship 

During  the  process  of  defining  our  roles  practices  and  procedures  for  the  course  and  the  mentoring  process, 
we  began  by  clarifying  our  assumptions  about  the  power  relationship.  The  first  theme  deals  with  this  power 
relationship  between  the  mentor  and  protege.  Specifically,  we  looked  at  how  power  was  experienced  and  how  was  it 
shared  within  this  specific  relationship.  The  power  relationship  movement  in  mentoring  is  gravitating  toward 
“democratic”  or  shared  power  structure  (Gunn,  1995;  Darwin,  2000).  Darwin  (2000)  posited  that  this  type  of 
mentoring  allows  the  mentor  and  protege  to  “share  intellectual  and  emotional  resources”  (p.  208).  A shared-power 
structure  using  a “developmental  network  perspective”  proposed  by  Higgins  and  Kram  (2001)  indicated  that 
preparing  individuals  for  the  world  of  work  requires  developing  skills  to  work  in  a network  capacity  of  mentorship 
instead  of  relying  upon  one  mentor  as  a superior  source  of  knowledge.  This  study  investigated  shared  power  in  the 
mentor/protege  relationship  in  a democratic  structure  to  prepare  the  proteges  for  the  world  of  work  in  an  online 
course. 

The  power  was  initially  shared  by  the  mentor  through  the  elevation  of  the  status  of  proteges  from  teaching 
assistants  to  co-instructors.  This  decision  was  not  a presumptive  one;  in  fact,  it  was  an  outcome  of  the  process 
whereby  the  mentor  and  proteges  defined  their  roles.  The  proteges  were  treated  as  equals  in  the  facilitation  of  the 
course  and  presented  to  the  students  within  the  course  as  a co-instructor,  both  at  the  initial  meeting  and  throughout 
the  course.  The  process  of  defining  our  roles  and  power  in  this  relationship  unfolded  over  the  weeks  preceding  the 
start  of  the  course  and  the  first  several  weeks  of  the  course.  After  the  identification  of  roles  overall,  the  mentor  and 
protege  began  to  work  on  the  characterization  of  those  roles  and  what  they  would  entail  in  the  context  of  the  course 
contents. 

During  this  characterization  the  proteges  were  allowed  much  flexibility  in  shaping  their  roles  as  co- 
instructors, only  taking  on  as  much  in  course  facilitation  as  they  felt  comfortable  with  and  selecting  specifically  the 
course  content  areas  they  would  oversee  as  instructors.  In  the  end,  this  specific  process  resembled  a balanced 
expertise:  by  providing  the  flexibility  to  the  protege  and  allowing  him/her  to  define  their  role  as  co-instructors  (e.g. 
being  allowed  to  choose  which  course  content  topics  they  felt  comfortable  in  overseeing)  and  in  doing  so  they  were 
allowed  to  focus  on  their  role  and  the  level  of  content  expertise  for  the  facilitation  of  that  course  content  and,  more 
specifically,  facilitation  of  those  discussion  topics  and  projects  for  grading. 

Related  to  this  was  the  process  of  Peer  Observations,  wherein  the  proteges  observed  initially  (during  the 
first  several  weeks  of  the  course)  as  the  mentor  facilitated  discussions  within  the  course.  Accompanying  this  process 
were  discussions  between  the  mentor  and  proteges,  focusing  on  how  to  facilitate  and  reasons  for  specific  facilitation 
processes,  leaving  room  for  the  proteges  to  experiment  to  some  degree  (without  taking  away  from  the  rigor  of  the 
discussions  or  the  course).  Moreover,  during  the  times  when  the  proteges  were  serving  as  instructors  and  facilitating 
the  discussions,  the  mentor  had  the  opportunity  and  time  to  observe  the  proteges'  practices.  This  allowed  for  further 
discussions  between  the  mentor  and  proteges  about  what  worked,  what  didn’t  work,  possible  reasons  for  these 
perceptions  and  best  practices.  Finally,  the  reflective  practices  employed  informally  by  the  mentor  and  proteges 
allowed  the  researchers  to  reflect  on  the  effectiveness  of  the  necessary  components  in  a mentoring  relationships  such 
as  these. 

Dispelling  Myths  About  Teaching  Online:  Changing  our  Beliefs 

The  results  from  the  field  texts  indicate  that  there  is  much  that  can  and  should  be  done  to  help  our  graduate 
students  learn  the  art  of  teaching  online,  especially  as  it  is  likely  they  will  find  themselves  teaching  online  if  they 
enter  academia.  For  example,  Hans  had  expressed  his  assumptions  about  teaching  online  prior  to  the  experience 
itself.  As  Hans,  a protege,  explained: 

1 think  initially  I thought  that  online  learning  would  be  much  cleaner  and  easier  to  manage  because  it  would 
be  in  digital  form,  everyone’s  comments  there  up  on  screen  to  see.  I also  thought  it  would  take  less  time  to 
manage  as  a course  than  a traditional  course,  (or  the  same  time,  because  of  my  reading  about  online 
courses)  or  at  least  it  would  be  more  convenient,  because  I could  do  most  of  the  work  from  home  and  had 
fewer  time  constraints.  I found  that  it  was  not  that  simple.  If  I thought  it  would  take  less  time  than  a 
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traditional  course,  I found  that  it  may  have  taken  more.  In  a regular  class  everyone  can  talk,  take  turns 
speaking  about  a topic,  asking  questions.  The  speech  comes  out,  it’s  analyzed  in  my  mind  and  if  I don’t 
understand  I ask  them  to  say  it  again,  or  vice-versa,  but  it  all  happens  so  quickly.  Online  everything  took 
longer.  It  took  longer  to  read  the  posts,  read  people’s  emails,  answer  their  questions,  try  to  make  myself 
clear  about  what  I meant,  clear  up  confusion,  and  gather  the  resources.  Even  with  everything  inside  of 
WebCT,  it  felt  like  the  class  began  to  sprawl  out,  with  all  of  the  posts  and  emails. 

Similarly,  the  mentoring  relationship  and  reflective  practices  embedded  served  to  allow  the  mentor  to 
revisit  her  beliefs  and  assumptions. 

Jennifer,  the  mentor,  wrote: 

I think  that  sometimes  we  as  instructors  tend  to  fall  into  a rut  in  terms  of  teaching  practices  and  activities 
we  engage  in,  whether  it  is  face-to-face  or  online  teaching.  I also  think,  based  on  speaking  with  other 
instructors  that  most  of  us  tend  to  teach  the  way  we  are  taught,  to  some  degree  at  least.  Having  other 
opinions  and  insights  about  tools  and  activities  made  me  step  out  of  my  own  comfort  zone  and  learn  to  use 
new  tools  and  set  up  group  discussions  and  activities  differently.  As  a previous  student  of  online  learning  I 
know  that  those  experiences  as  a student  were  valuable,  but  I also  had  to  realize  that  in  the  world  of  online 
courses  expectations  and  emerging  technologies  are  evolving.  Given  this,  I was  able  to  have  discussions 
with  the  proteges  about  what  might  enhance  the  students'  learning,  making  it  more  effective  and  efficient. 
This  also  came  into  play  in  terms  of  course  organization.  In  my  mind,  the  online  course  home  page  in 
WebCT  was  very  intuitive,  but  not  everyone  saw  it  that  way. 

In  the  end  the  proteges  found  that  there  was  much  more  to  consider  than  they  imagined  while  the  mentor 
was  reminded  of  what  students'  expectations  were  in  terms  of  organization  and  online  activities. 

Communication  and  Presence  in  Online  Environments 

The  researchers,  as  any  instructors  of  online  courses  would  do,  looked  to  the  best  methods  for  getting  their 
presence  across.  The  mentor,  Jennifer,  has  been  conducting  ongoing  research  into  the  social  aspects  of  online 
learning,  including  social  presence;  therefore,  this  topic  was  one  that  was  sure  to  arise.  Jennifer,  the  mentor,  wrote: 

Given  my  previous  experiences  as  a student  in  online  courses  1 had  come  away  with  several  ideas  about 
how  to  make  a course  better,  more  interactive  both  in  terms  of  activities  and  discussion,  how  to  make  each 
student  feel  comfortable  in  an  online  environment.  Most  importantly,  I learned  that  the  presence  of  the 
instructor  is  of  utmost  importance  in  terms  of  not  only  student  satisfaction  but  potentially  student  outcomes; 
students  want  adequate  and  prompt  feedback.  I feel  that  this  area  of  online  teaching  is  one  of  the  most 
important  yet  I also  find  that  it  is  the  one  that  perhaps  takes  the  most  practice.  Related  to  this  is  act  of 
helping  your  students  understand  and  embrace  their  own  presence  as  a classroom  management  strategy.  In 
other  words,  if  students  are  aware  of  their  presence  they  can  disagree  with  a peer  online,  yet  do  so  without 
coming  across  as  being  hostile,  a common  complaint.  Moreover,  1 think  it  takes  practice  as  you  shift  from  a 
student  in  an  online  discussion  to  an  authority  figure,  such  as  an  instructor.  Y our  persona,  and  therefore 
your  presence,  also  needs  to  shift. 

Hans,  a protege,  wrote: 

Confusion  is  not  as  easy  to  dispel  online  - you  can’t  do  the  quick  Q+A  that  usually  leads  to  resolution  of  a 
problem.  Also,  I was  much  more  aware  of  the  limitations  of  my  own  writing.  I tried  to  be  much  more  clear 
than  usual  - I almost  never  put  up  a post  without  editing  it  and  re-reading  it  at  least  once,  making  changes 
to  try  and  maintain  a friendly,  open,  knowledgeable  online  personally. . .1  feel  so  much  more  comfortable 
doing  classes  where  I feel  I can  tell  whether  a student  gets  it  or  not  by  looking  at  their  faces  or  gauging 
their  reaction  time.  That’s  much  more  difficult  online  - it’s  the  difference  between  having  a discussion 
about  something  using  the  natural  people  skills  one  has  developed  over  the  years,  and  teaching  class  by 
writing  short  letter  to  each  student  and  reading  all  of  theirs. 
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Ann,  a protege,  wrote: 

The  biggest  challenge  for  me  in  shifting  from  teaching  face-to-face  (F2F)  for  decades  to  being  a co- 
instructor  for  the  first-time  in  an  online  class  was  dealing  with  the  lack  of  the  nonverbal  cues  (which  1 so 
heavily  rely  upon).  1 use  nonverbal  cues  constantly  to  help  get  across  my  points  with  gestures,  actions,  and 
expressions  in  the  F2F  classroom.  I also  carefully  observe  the  nonverbal  cues  of  my  students  to  help 
determine  how  I can  facilitate  the  learning  environment  better.  It  was  like  having  your  hands-tied  when 
functioning  in  an  online  discussion!  Jennifer  [my  mentor]  helped  alleviate  my  anxieties  about 
communicating  without  the  class  by  conveying  it  was  a process  that  would  improve  with  time.  By  serving 
as  a co-instructor,  she  provided  me  with  the  shared  power  necessary  to  get  the  “real  feel”  for  how  other 
pedagogical  methods  and  strategies  can  be  used  to  find  out  what  normally  would  be  gained  through 
nonverbal  cues. 

By  providing  the  opportunity  to  discuss  prior  to  developing  and  teaching  their  own  online  courses,  the 
proteges  left  the  experience  feeling  more  prepared  in  terms  of  techniques  and  what  to  expect  in  terms  of  presence 
and  the  role  it  plays  in  an  effective  course. 

Managing  the  Online  Course:  Not  What  One  Would  Expect 

Course  design  and  management  calls  for  variations  on  traditional  techniques  and  methods.  Anyone  who 
taught  online  has  probably  come  to  realize  that  students  in  those  courses  tend  to  need  very  explicit  guidelines  and 
expect  immediate  attention.  The  mentor  and  proteges  discussed  these  issues  both  in  advance  and  as  they  arose 
during  the  course. 

Hans,  a protege,  wrote: 

Helpful  guidance  on  practical  online  course  management  was  invaluable.  Questions  came  up  that  I hadn’t 
thought  about  before  teaching  - when  so  many  posts  are  written  a week,  how  often  would  I go  in  and  look 
at  them?  My  mentor  suggested  twice  a week,  spaced  out.  Another  question  comes  with  that  one  - when  I 
go  into  the  discussion  boards,  how  strongly  do  I make  my  presence  known?  How  many  posts  do  I reply  to? 
If  I reply  to  every  one,  then  it  may  be  expected  every  week,  even  if  it’s  not  required.  In  addition,  depending 
on  how  the  students  view  the  instructor,  it  may  be  seen  as  the  final  word  on  the  topic  (so  often  comments 
had  to  take  into  account  the  content  of  the  previous  comment  and  deliberately  drive  the  discussion  forward 
with  another  question).  Another  thing  1 hadn’t  planned  on  was  how  long  it  would  take  to  compose 
discussion  posts  and  emails  - just  so  I was  sure  I was  clear,  positive,  and  not  taken  the  wrong  way.  As  far 
as  the  design  goes,  this  course  took  an  interesting  approach  in  engaging  the  students  as  facilitators  on  a 
discussion  - after  the  first  few  weeks  the  discussions  questions  and  control  of  discussions  was  given  to  a 
pair  of  students,  who  selected  the  readings  for  the  week  based  on  a topic  of  interest  to  them.  So,  while  the 
mentor  modeled  for  me  initially,  we  moved  to  modeling  for  the  students. 

Ann,  a protege,  wrote: 

Managing  people  is  usually  not  a problem  for  me.  But,  as  I began  the  online  teaching  adventure,  I 
wondered,  “How  can  the  content  be  best  organized  to  make  it  user-friendly  and  intuitive?”  “Also,  how  can 
we  get  students  to  focus  on  the  content  from  the  beginning  of  a module  with  a particular  starting  point?” 
My  protege  role  allowed  me  to  observe  a section  Jennifer  uses  to  accomplish  both  focus  on  the  content-at- 
hand  and  provide  an  intuitive,  user-friendly  interface.  The  first  document  in  the  module  is  called 
“Something  to  Think  About”.  Another  thing  she  used  within  each  module  as  well,  as  in  the  syllabus,  is  a 
module  assignment  list.  These  two  items  will  definitely  be  part  of  online  courses  I design,  because  they  are 
beneficial  to  the  learner  and  help  the  learner  see  tasks  as  more  manageable. 

Jennifer,  the  mentor,  wrote: 

It  has  taken  me  much  practice  to  learn  what  works  best  for  the  students  in  terms  of  feedback  and  presence 
and  how  they  fit  in  with  the  class  management  and  design.  As  Hans  explained  about  setting  the  guidelines 
into  place  about  selecting  specific  days  we  would  post,  when  to  expect  to  hear  back  from  us,  and  even 
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explaining  what  we  perceived  our  role  in  the  discussions  to  be,  these  techniques  can  help  an  instructor  and 
the  students  keep  their  sanity.  I also  know  that  each  class  is  different  and  will  have  different  days  they  are 
active  on.  When  it  comes  to  discussions  I tend  to  wait  a few  weeks  as  I delineate  their  patterns  and  then 
revise  the  days  I'll  be  part  of  the  ongoing  discussions.  For  example  this  semester  my  students  are  not 
posting  until  the  fourth  day  of  seven  for  the  most  part,  whereas  in  other  courses  that  wouldn’t  hold  true.  I 
want  to  be  there  when  I am  most  needed,  to  be  effective.  Keeping  a line  of  communication  open  to  the 
students  is  very  high  priority  for  me. 

Perhaps  one  of  the  biggest  lessons  learned  by  the  two  proteges  was  that  it  is  okay,  even  necessary,  to  say  to 
students,  "This  is  what  you  can  expect  and  I will  not  be  in  the  course  7 days  a week."  Similarly,  proteges  learned 
that  there  is  no  way  to  please  all  of  the  students  in  each  course,  something  that  appears  more  transparently  in  online 
courses.  Online  students  tend  to  be  more  free  with  their  comments,  both  positive  and  negative,  perhaps  due  to  the 
space  provided  by  the  medium. 


Implications 

Through  the  analysis  of  the  field  texts,  the  practices  and  processes  for  mentoring  a graduate  student  in 
pedagogical  practices  for  online  learning  several  themes  emerged,  both  pedagogical  and  procedural.  Following  are 
eight  guidelines  developed  by  the  mentor  and  proteges  for  others  interested  in  this  process.  While  a mentor  or 
protege  could  use  the  guidelines,  they  are  phrased  mainly  for  the  mentor. 


1 

Pre-Assumptions 

Discuss  pre-assumptions  about  teaching  in  an  online 
environment.  What  are  the  myths  and  the  realities? 

2 

Expectations 

Discuss  expectations  with  proteges  about  what  they  want/need 
from  the  experience,  mentor  may  provide  additional 
possibilities. 

3 

Power  Relationship 

Based  on  the  proteges  needs,  establish  the  power  relationship, 
preferably  prior  to  beginning  the  experience  so  that  the  power 
relationship  and  balance  will  be  evident  to  the  students  (e.g. 
they  will  see  the  protege  as  an  authority  figure  versus  a peer). 

4 

Designate  course  roles 

After  establishing  expectations  and  roles  (power  relationship), 
designate  what  roles  the  mentor  and  protege  will  have  within  the 
course  (e.g.  designate  content/topics  to  be  developed,  grading 
responsibilities,  leadership  roles  for  activities). 

5 

Model  Instruction 

Mentors  should  begin  by  modeling  their  instructional  techniques 
and  methods  for  the  proteges.  The  modeling  process  should  also 
be  discussed  so  that  the  protege  doesn’t  only  see  what  is  being 
taught  and  how  it  is  being  taught,  but  why  it  is  being  taught  in  a 
particular  manner. 

6 

Discuss  Best  Practices 
in  Course  Management 

From  early  on  in  the  experience  it  will  be  important  for  the 
proteges  to  have  a grasp  of  course  development  (e.g.  lead  time 
for  developing  modules,  organization  and  the  need  for  multiple 
drop  spots),  course  management  (setting  online  hours,  setting 
guidelines  for  students  in  terms  of  activities  and  especially 
online  discussions,  clearly  outlined  student  expectations. 

7 

Reflection  and 
Discussion 

Both  the  mentor  and  protege  should  keep  an  informal  journal 
that  records  expectations,  things  to  discuss,  reasoning  behind 
particular  practices,  and  questions  arising  form  the  process.  It 
will  be  important  for  the  mentor  to  allow  the  protege  to  feel 
comfortable  asking  questions  such  as  "why  did  you  do  this"  and 
take  it  as  a learning  point  rather  than  criticism. 

8 

Challenging  Beliefs 

As  the  experience  progresses,  discussion  that  focus  on  not  only 
the  changes  of  the  proteges  beliefs  but  also  insights  provided  to 
the  mentor  are  invaluable.  Part  of  the  mentoring  process  should 
be  that  all  parties  are  coming  away  with  more  knowledge  and 
expertise. 
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Implications  are  broad,  especially  as  the  number  of  online  courses  continues  to  grow,  and  the  number  of 
faculty  being  asked  to  teach  online  parallels  that  growth.  Graduate  students  headed  for  a profession  related  to 
teaching/instruction  should  posses  the  confidence  and  ability  to  teach  online  and  it  is  the  responsibility  of  graduate 
programs  to  provide  real-world  experiences  for  them,  just  as  they  would  be  engaged  in  similar  experiences  with 
teaching  in  traditional  classes.  Obviously  there  are  some  limitations  to  this  approach,  especially  as  many  faculty  are 
considered  novices  themselves  in  the  area  of  online  teaching,  and  the  opportunities  for  mentoring  graduate  students 
in  teaching  online  courses  are  narrow.  However,  providing  such  opportunities  does  help  both  mentors  and  proteges 
to  be  better  instructors  as  they  reflect  on  the  process.  How  often  do  we  as  instructors  implement  an  activity  without 
fully  considering  the  implications  of  changing  the  tool  or  modifying  the  activity,  thereby  making  it  more  effective? 
This  process  simply  asks  us  to  revisit  our  reasons  for  doing  so. 

Just  as  student  teachers  are  not  able  to  “to  begin  to  develop  their  new  body  of  practical  professional 
knowledge  until  they  enter  the  classroom”  (Furlong  and  Maynard,  1995),  graduate  students  do  not  develop  their 
practical  teaching  experiences  until  they  enter  the  classroom — and  this  includes  not  only  traditional  classrooms  but 
online  “classrooms”  as  well.  It  appears  the  online  teaching  environment,  as  discussed  earlier  often  new  to  the 
graduate  student  protege,  requires  guidance  and  support,  which  is  well  documented  in  teacher  education  as  we 
prepare  K-12  teachers  (Cameron- Jones  & O'Hara,  1997).  Furlong  and  Maynard,  who  base  their  conceptualizations 
of  learning  to  teach  on  Schon’s  work  (1987),  provide  us  with  insight  into  mentoring  student  teachers,  a strong 
parallel  to  graduate  students'  learning  to  teach.  In  their  mentoring  process  they  describe  coaching  through  a 
reflective  approach,  one  in  which  the  mentor  takes  on  several  roles:  the  mentor  models  appropriate  strategies  (e.g. 
rules,  routines,  establishing  authority),  the  mentor  coaches  students  in  their  teaching  competencies  (e.g.  provides 
feedback,  facilitates  reflection-on-action),  the  mentor  serves  as  a critical  friend  (e.g.  observes  student/protege,  re- 
examines instructional  design),  and  the  mentor  serves  as  co-inquirer  (e.g.  partnership  teaching)  (Furlong  & 

Maynard,  1995,  p.  181).  Moreover,  as  Furlong  and  Maynard  explain,  it  is  important  for  mentors  to  flexible  in 
adapting  to  individual  protege’s  needs  and  levels  (1995,  p.  180).  In  fact  in  collaborative  teaching,  where  the  protege 
is  involved  in  the  planning,  teaching  and  evaluation  of  the  lesson,  the  protege  has  the  opportunity  to  model  the 
teachers’  [mentor’s]  teaching  at  a level  of  great  detail  (Furlong  & Maynard,  1995,  p.  180).  Similarly,  examples  of 
basic  mentoring  activities  provided  by  Anderson  and  Shannon  (1988)  include  demonstrating  teaching  techniques  to 
a protege,  observing  the  protege’s  classroom  teaching  and  providing  feedback,  and  holding  support  meetings  with 
the  protege  (p.  41). 

Finally,  the  guidelines  provided  may  also  have  applications  in  other  settings,  including  teacher  education  or 
business,  or  other  areas  in  which  a mentor  and  protege  would  work  closely  together.  The  content,  in  this  case 
teaching  in  an  online  environment,  might  be  replaced  with  other  complex,  time-consuming  processes.  For  example, 
in  education,  processes  such  as  dealing  with  parents  or  student  discipline,  or  in  business,  complex  business  practices 
such  as  managing  client  accounts  and  projects.  The  important  cyclic  steps  of  defining  roles  and  examining  their 
duties,  and  creating  or  revising  desired  outcomes  still  apply  to  these  scenarios  and  the  clarification  of  roles  and 
outcomes  may  ease  the  transition  to  more  professional  status  for  the  protege. 
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Introduction 

Computer-mediated  communication  (CMC),  specifically  computer  conferencing,  has  captured  the  interest 
of  educators  and  psychologists  especially  in  developing  collaborative  and  participatory  learning  communities  to 
promote  higher-order  conceptual  learning  and  analytical  thinking  skills,  and  to  support  social  construction  of 
knowledge  (Curtis  & Lawson,  2001;  Harasim,  2000;  Kanuka  & Anderson,  1998).  Computer  conferencing  has  the 
capability  of  creating  flexible  communication  patterns  (Berge  & Collins,  1996),  removing  time  and  space 
restrictions,  promoting  self-reflective  dialogues  (Harasim,  1993),  and  creating  a (discussion)  forum  for  the 
construction  of  knowledge  in  the  teaching  and  learning  process  (Heller  & Kearsley,  1996;  Henri,  1992). 

Computer  conferencing  can  provide  intellectual  on-going  discussion  focusing  on  a topic  or  relevant 
concepts  within  the  course.  In  addition,  it  allows  the  learner  to  participate  in-depth  discussions  by  checking 
references,  referring  back  to  preceding  topics  and  taking  any  amount  of  time  to  prepare  a detailed  comment  or 
argument.  Furthermore,  computer  conferencing  can  be  used  to  facilitate  group  project  work  and  encourage 
interactive  participation  and  interpersonal  collaboration  between  instructor  and  learners  and  among  learners. 

Computer-mediated  conferencing,  in  particular  asynchronous  online  discussion,  is  becoming  increasingly  a 
common  instructional  strategy  in  higher  education  because  of  its  potential  to  support  social  interaction,  collaborative 
learning,  knowledge  building,  and  critical  thinking.  Despite  its  popularity,  there  are  few  theories  and  limited  amount 
of  empirical  evidence  for  the  claims  made  about  the  potential  benefits  of  this  medium  (Henri,  1992;  Mason,  1992; 
Gunawardena  et  ah,  1997;  Garrison  et  ah,  2001).  There  is  still  a lack  of  clarity  of  the  dynamics  of  online  discussion 
and  how  it  may  be  utilized  to  foster  students’  cognitive  development  (Hara  et  ah,  1998)  There  are  few  theories  and 
little  empirical  research  on  what  online  collaboration  or  social  interaction  is  and  should  be;  how  online  discussions 
promote  learning  experience,  in  particular,  higher-order  thinking  and  knowledge  construction.  Thus,  in  support  of 
research,  this  study  was  undertaken  to  scrutinize  the  patterns  of  student  interaction  taking  place  in  online  discussions 
and  to  examine  the  co-construction  of  knowledge  among  them.  In  addition,  the  study  was  aimed  at  guiding 
researchers  and  educators  in  the  design  and  utilization  of  the  efficiency  of  CMC  that  could  increase  collaboration 
and  support  learning. 


Research  Objectives 

The  adoption  of  computer  conferencing  for  learning  is  still  relatively  under-researched.  In  spite  of 
considerable  research  interest  in  this  area,  there  still  remains  a need  for  examining  the  best  use  of  the  richness  and 
efficiency  of  exchange  provided  by  CMC  content.  It  is  important  to  have  a better  understanding  of  the  potentials  of 
CMC  in  learning  experience.  The  primary  objective  of  this  study  was  to  analyze  the  value  and  quality  of  the 
asynchronous  online  discussion  for  learning  experience  and  knowledge  construction  through  social  interaction. 
Specifically,  the  aim  was  to  investigate: 

■ the  social  participation  and  interactive  patterns  in  asynchronous  online  computer  conferences  characterized 
by  explicit,  implicit,  or  independent  interaction 

■ whether  the  interactive  nature  of  CMC  exchange  support  and  encourage  the  development  of  knowledge  and 
practice  of  higher-order  thinking  skills  through  collaborative  learning  process 

■ pedagogical  factors  that  may  have  influence  on  the  level  of  knowledge  building  and  the  type  of 
collaborative  learning  discourse 


Theoretical  Framework 

The  important  assumption  of  constructivism  is  that  knowledge  is  actively  constructed  based  upon  unique 
set  of  experiences  and  beliefs.  In  constructivism,  knowledge  is  always  under  construction  as  a cumulative  history  of 
interactions  in  authentic  and  meaningful  contexts.  Radical  constructivism,  a form  of  constructivist  theories,  claims 
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that  knowledge  is  the  individual  cognitive  process  of  the  human  mind,  or  the  function  of  cognition  that  serves  the 
organization  of  the  experiential  world.  The  world  is  constructed  in  minds  through  the  construction  of  personal 
realities  or  experiences.  The  other  prevalent  position  in  constructivist  learning  theories-social  constructivism-  claims 
that  learners  obtain  knowledge  and  make  sense  of  their  experiences  through  communication,  or  language,  as  they 
explore  new  perspectives  and  communicate  their  understanding  with  others  (Jonassen,  et  al.,  1995;  Hein,  1991; 
McGuire,  1996;  Warschauer,  1997).  From  a social  constructivist  viewpoint,  learner  is  actively  acquiring  the 
knowledge  through  language  in  social  experiences.  Vygotsky  (1978),  most  often  associated  with  social 
constructivism,  emphasized  the  effect  of  social  experiences  that  occur  over  time  in  a contextual  and  situated 
synthesis  on  knowledge  construction.  In  other  words,  knowledge  is  constructed  through  social  interaction  and 
interpersonal  communication  based  upon  interpretation  of  information  and  learning  experience  within  a context. 

Computer  conferencing  or  online  discussions  can  be  used  to  encourage  interactive  participation  and 
interpersonal  collaboration  between  instructor  and  learners  and  among  learners.  Collaboration  among  learners 
enhances  the  quality  of  learning  and  formulates  a deeper  understanding  of  the  content.  Jonassen  et  al.  (1995)  notes 
that  "learning  is  necessarily  a social  dialogical  process  in  which  communities  of  practitioners  socially  negotiate  the 
meaning  of  phenomena"  (p.9).  However,  designing  and  maintaining  a learning  environment  (e.g.,  asynchronous 
online  discussions)  in  which  knowledge  is  constructed  through  social  negotiations  is  challenging.  It  is  essential  to 
understand  the  way  interactive  discussions  through  computer  mediated  communication  support  the  co-construction 
of  knowledge. 

Gunawardena,  Lowe,  & Anderson  (1997)  developed  an  Interaction  Analysis  Model  - serves  as  the 
theoretical  framework  for  this  study-  to  specifically  examine  “the  negotiation  of  meaning  and  co-construction  of 
knowledge  in  collaborative  learning  environments  facilitated  by  computer  conferencing”  (Gunawardena  et  al.,  1997, 
p.  397).  Interaction  Analysis  Model  is  based  on  constructivistic  conceptions  of  learning.  Gunawardena  et  al.  (1997) 
addresses  two  kinds  of  knowledge  creation  that  take  place  in  any  shared  learning  experience,  the  “individual”  and 
the  “social.”  They  recognize  the  interdependence  of  both  the  individual  and  social  construction  of  knowledge  in 
their  model.  They  developed  this  model  from  a constructivist  perspective  by  using  the  metaphor  of  a patchwork  quilt 
block  to  describe  the  significance  of  interaction.  Within  this  model,  the  notion  of  interaction  in  a computer-mediated 
conference  is  defined  as  the  production  of  new  knowledge  and/or  the  understanding  of  new  meaning.  The  model 
proposes  a five  phase  evolution  of  knowledge  construction:  Phase  I)  sharing/comparing  of  information;  Phase  II) 
discovery  and  exploration  of  dissonance  or  inconsistency  among  ideas,  concepts  or  statements;  Phase  III) 
negotiation  of  meaning/co-construction  of  knowledge;  Phase  IV)  testing  and  modification  of  proposed  synthesis  or 
co-construction;  Phase  V)  agreement  statement(s)  /applications  of  newly-constructed  meaning. 

Methodology 


Context 


The  data  for  this  study  were  collected  over  a 14-week  semester  from  an  online  master’s  course  offered  by 
the  college  of  education  at  a large  Midwestern  U.S.  university,  designed  for  K-12  teachers  and  other  educational 
practitioners.  The  course  management  tool,  WebCT,  was  the  main  course  component  and  primary  communication 
medium  among  students.  Discussion  board  of  WebCT  was  utilized  for  the  majority  of  class  work  and  learning 
experience  of  students.  Participation  to  weekly  discussions  at  least  three  times  was  required.  The  conference 
transcripts  of  three  main  discussion  topics  were  selected  for  this  study  to  analyze  the  shared  learning  experience 
through  online  discussions  of  WebCT. 

Subjects 


There  were  15  students  enrolled  in  the  course.  Two  were  undergraduate  students,  eleven  of  them  were 
masters,  and  two  of  them  were  doctoral  students.  60%  of  the  students  were  working  full  time,  33.3%  were  working 
part-time  and  6.7%  were  not  working.  The  majority  of  students  were  adult  learners  with  an  average  age  of  29  and 
73%  of  them  working  as  a teacher  or  an  educator  in  different  fields  such  as  elementary  school  computer  teacher, 
secondary  school  English  teacher,  social  studies  teacher,  community  college  instructor,  and  higher  education  dean  at 
a community  college.  36%  of  the  students  had  never  taken  a distance  education  whereas  43%  of  them  took  more 
than  one  distance  course.  Therefore,  the  majority  of  students  were  experienced  in  learning  at  a distance. 
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Data  Collection  & Analysis 

In  this  study,  quantitative  and  qualitative  research  methodologies  were  conducted.  For  quantitative  modes 
of  inquiry,  survey  was  conducted  to  collect  demographic  and  descriptive  data.  The  survey  was  completed  by  14  of 
15  students  giving  a return  rate  of  93.3%.  Additionally,  for  the  first  research  objective  on  social  participation  in 
computer  conferencing,  descriptive  statistics  such  as  the  number  and  length  of  student  postings  contributed  each 
discussion  topic  were  calculated.  Moreover,  in  order  to  understand  the  underlying  patterns  of  interaction  in  the 
structure  of  the  course,  Howell-Richardson  and  Mellar’s  (1996)  interaction  maps  that  provide  visual  representations 
of  electronic  conferencing,  and  Henri’s  (1992)  criteria  related  to  message  interactivity  (i.e.,  explicit,  implicit,  or 
independent  statements)  were  combined  to  create  conference  activity  graphs  to  illustrate  the  flow  of  the  discussions 
and  the  direction  of  the  postings.  The  unit  of  analysis  for  those  was  selected  as  the  “unit  of  message”. 

For  qualitative  part,  interview  and  content  analysis  methodologies  were  conducted.  The  course  instructor 
was  interviewed  to  obtain  background  information  about  the  course  (i.e.,  learning  objectives,  design  strategy,  and 
assessment)  and  her  perceptions  on  student  learning  experience.  Content  analysis  was  chosen  as  the  main 
methodology  of  this  study  to  analyze  the  computer  conference  transcripts  to  examine  the  co-construction  of 
knowledge  in  computer  conferencing.  All  participant  names  were  replaced  with  pseudonyms  in  conference 
transcripts  to  assure  the  confidentiality.  To  answer  the  second  research  question,  the  content  of  the  messages  of  three 
discussion  topics  was  coded  based  on  Gunawardena  et  al.’s  (1997)  interaction  analysis  model  using  computer 
assisted  qualitative  data  analysis  package,  Atlas. ti. 

According  to  Gunawardena  et  al.  (1997),  a message  as  a whole  embodies  a student’s  cognitive  activity  and 
contribution  to  the  construction  of  knowledge.  Thus,  they  used  the  complete  messages  in  discussion  forum  as  the 
unit  of  analysis.  However,  the  unit  of  message  did  not  fit  in  this  study  because  some  messages  contained  very  little 
information;  others  contained  three  or  more  distinct  ideas,  comments,  complex  arguments  or  hypotheses  addressing 
different  concepts  or  questions  raised  during  the  discussions.  Therefore,  the  unit  of  analysis  was  selected  as  the  unit 
of  thematic  (meaning).  Henri  (1992)  justifies  this  type  of  unit  of  analysis  by  arguing  that  “it  is  absolutely  useless  to 
wonder  if  it  is  the  word,  the  proposition,  the  sentence  or  the  paragraph  which  is  the  proper  unit  of  meaning,  for  the 
unit  of  meaning  is  lodged  in  meaning”  (p.  134). 

Gunawardena  et  al.’s  framework  was  slightly  modified  to  make  it  more  relevant  and  apparent  for  coders.  It 
was  also  modified  because  of  the  differences  between  thematic  unit  of  analysis  and  complete  message  unit  of 
analysis  used  by  Gunawardena  et  al.  (1997).  The  primary  coder  analyzed  the  data  according  to  this  framework  on 
three  separate  occasions  to  validate  the  coding  procedures  of  the  modified  model.  Descriptive  rules  along  with 
examples  for  the  coding  process  were  explained  to  two  other  coders  in  a training  session.  Then,  two  coders  analyzed 
a portion  of  the  data  (10%  of  total)  independently.  The  final  inter-coder  agreement  was  85%. 

Findings  & Discussions 


Participation  Results 

Overall  participation  results  in  the  course  are  as  follows:  Students  posted  a message  on  the  discussion 
forum  per  week  approximately  2 times.  In  order  to  post  a message,  they  spent  25  minutes  on  average.  In  addition, 
students  replied  to  2 messages  on  average  in  a week. 

Table  1 below  provides  an  overview  of  participation  levels  for  each  discussion  topic  area.  In  term  of  the 
discussion  topic,  Online  Educator,  7 out  of  the  15  students  contributed  to  the  discussion  in  about  one-month  period. 
The  average  number  of  posts  per  student  was  2.  For  Attitude,  8 out  of  15  students  in  the  class  participated  in  the 
discussion  in  about  three-month  period  with  an  average  number  of  2.4  posts  per  student.  7 out  of  15  students 
contributed  to  the  discussion  topic.  Assessment,  in  40-day  period.  The  average  number  of  messages  per  student  was 
2.6. 
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Table  1.  Participation  by  Topic 


Discussion 

Topic 

Discussion 

Period 

Total  # of 
students 
participated  out 
of  total  # 
enrolled 

Total  # of 
messages 

Total  # of 
instructor 

messages 

Average  # 
of  posting 
per 

student 

Total  # of 
words 

Online 

Educator 

September20- 

Octoberl9 

7 of  15 

14 

0 

2 

2266 

Attitude 

September  18- 
December6 

8 of  15 

19 

0 

2.4 

1941 

Assessment 

November2- 

DecemberlO 

7 of  15 

18 

4 

2.6 

2441 

Total/Average 

22/7.3 

51/17 

4/1.3 

2.3 

6648/2216 

Findings  revealed  that  the  level  of  overall  participation  in  the  course  and  the  participation  to  three 
discussion  topics  was  low.  The  statement  made  by  a student  to  an  open  ended  survey  question  described  clearly  the 
learning  environment  in  terms  interaction  and  participation:  “I  need  people  to  respond  to  questions/comments.” 
However,  it  is  found  that  students  were  in  control  of  the  flow  of  discussions,  in  other  words,  they  dominated  the 
discussions,  not  the  instructor.  This  is  most  likely  because  of  one  of  the  course  goals:  to  create  a constructivist  and 
flexible  online  learning  environment  for  students  to  develop  cognitively  demanding  knowledge  by  interacting  with 
their  peers.  The  instructor  was  purposefully  providing  a learner-centered  context  in  which  students  were  in  charge  of 
their  own  learning  by  responding  to  others  critically  and  thoughtfully.  It  is  also  found  that  instructor’s  contribution 
to  discussions  made  no  such  difference  in  terms  of  the  participation  level  of  students. 

Interaction  Patterns 

In  discussion  topic-  Online  Education,  the  discussion  was  quite  straightforward,  every  student  responded  to 
the  previous  comment.  It  was  found  that  student  postings  were  more  explicit  and  directly  referred  to  others  in  their 
content  when  there  was  a starter  and  facilitator  in  online  discussion.  In  discussion  topic  - Attitude,  every  message 
was  connected  either  explicitly  or  implicitly.  Unlike  in  Online  Educator,  some  postings  were  referenced  by  more 
than  one  student.  It  was  also  found  that  students  began  to  pay  attention  to  multiple  threads  as  the  discussion 
progressed  and  the  discussion  became  more  continuous  and  engaging  especially  in  the  later  stages.  In  comparison  to 
Online  Educator,  there  was  a synergistic  interaction  among  students  in  Attitude. 

The  Assessment  was  the  only  topic  of  the  three,  which  had  a teacher  presence  (see  Figure  1).  Instructor 
posted  a prompt  to  start  the  discussion  and  contributed  with  three  other  messages  during  discussion:  her  second 
message  was  another  prompt  that  was  just  posted  after  the  first  one  supporting  the  previous  one  with  explanations, 
examples  and  resources  where  students  can  find  information.  The  other  postings  of  instructor  were  initiating 
prompts  such  as  “Now  it  is  another  person's  turn  to  answer  a question”  or  “Also  think  about  informal  assessment  for 
your  ILO  Orientation,  what  and  how?”  All  posts  of  the  instructor  were  coded  as  Phase  I (sharing/comparing  of 
information).  Overall,  her  role  can  be  described  as  a triggering  role  because  of  providing  background  information 
that  culminates  a question. 

Discussions  in  the  Assessment  topic  mainly  followed  the  questions  or  comments  posed  by  key  participants. 
In  this  discussion,  the  key  participants  were  the  instructor  and  Barbara  (see  Figure  1).  Especially,  Barbara  played  a 
role  of  a facilitator  by  providing  arguments,  resources,  examples,  and  responds  to  others.  She  also  acted  as  if  a 
summarizer  of  the  messages  which  gave  more  structure  to  the  discussion  and  made  it  more  engaging.  For  instance, 
Barbara  (#11)  not  only  implicitly  responds  to  others  who  made  comments  about  her  previous  posts  but  also 
explicitly  responds  to  a peer  by  making  statement  of  agreement,  sharing  more  information,  and  finally  asking  more 
initiating  questions  to  carry  on  the  discussion. 
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2.  Instructor 

Prompt  B 

V " > 


j j : Initial  prompt 

: Participant  Posting 

— ► : Indicates  direction  of  explicit  posting 

— ► : Indicates  direction  of  implicit  posting 

i — i 

! I : Set  of  postings  replied  by  participant(s)  at  a time 

Figure  1.  Interaction  Pattern  of  Discussion  Topic  - Assessment 


Knowledge  Construction  Phases 

The  analysis  of  data  gathered  for  the  content  analysis  showed  that  the  predominance  of  messages  fell  into 
the  first  phase  of  co-construction  of  knowledge:  sharing  and/or  comparing  of  information  (see  Table  2).  For 
instance,  the  message  below,  which  was  coded  at  Phase  I,  exemplified  two  types  of  statements.  Maria,  in  her 
message,  provides  background  information  citing  a resource  along  with  her  personal  opinion,  observation  and 
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reflection.  Later,  message  concludes  by  asking  questions  to  trigger  a discussion  in  order  to  obtain  group  consensus 
on  the  new  information.  This  is  an  example  for  the  early  process  of  social  negotiation  of  knowledge  construction. 


As  I was  reading  chapter  3 it  struck  me  that  probably  the  most  important  thing  in  a distance 
program,  according  to  McVay  is,  "the  student's  ability  to  obtain  information  and  research 
materials."  (McVay,  54)  [PHI/A]  She  discusses  a scenario  where  3 students  are  taking  a distance 
course:  1 in  a rural  area,  1 in  the  Australian  Outback,  and  one  at  a University  campus.  [PHI/A] 
Our  situations  do  not  mirror  McVay's  theoretical  situation,  [PHI/ A]  but  we  are  all  spread  out; 
[PHI/ A]  some  of  us  at  ISU  and  some  of  us  in  rural  areas— while  some  of  us  are  just  so  busy  that 
it's  as  if  we're  in  the  Outback.  [PHI/ A] 

From  the  Library  Support  for  Distance  Education,  one  of  the  important  principles  they  believe 
should  be  incorporated  into  DE  is,  "Empowerment  of  students  to  access  information  and  to 
perform  their  own  research  on  a self-service  basis."  [PHI/ A] 

The  reason  why  I bring  this  point  about  empowerment  up  is  because  in  my  opinion  it  is  vital  that 
students  participating  in  DE  already  have  good  research  skills  and  then  they  also  need  access. 
[PHI/A]  So,  who  is  responsible  #1  to  make  sure  that  students  (of  all  ages)  have  the  research 
skills, [PHI/D]  and  #2  who  will  make  sure  that  students  have  access  (not  as  a product  of  students 
shelling  out  lots  of  cash)?  [PHI/D] 

There  is  so  much  research  on  equity  in  access  that  I'm  sure  some  of  our  professors  have  had  a hand 
in  researching.  [PHI/A]What  are  everyone's  thoughts?  [PHI/D] 


Table  2.  Knowledge  Construction  Phases  by  Topic 


Topic 

Phase  I 

Phase  II 

Phase  III 

Phase  IV 

Phase  V 

n 

% 

n 

% 

n 

% 

n 

% 

n 

% 

Online  Educator 

106 

72 

12 

8 

20 

13 

9 

6 

i 

1 

Attitude 

46 

40 

23 

20 

30 

26 

16 

13 

i 

1 

Assessment 

123 

82 

2 

1 

18 

12 

6 

4 

i 

1 

Total/ Average 

275 

66 

37 

9 

68 

16 

31 

8 

3 

1 

On  the  other  hand,  several  exchanges  showed  evidence  of  movement  from  Phase  I to  higher  phases.  Of  the 
three  discussion  topics,  Attitude  had  the  highest  percentage  of  Phase  II  (20%),  Phase  III  (26%),  and  Phase  IV  (13%). 
The  evidence  of  higher  phases  was  observed,  in  particular,  when  the  discussion  became  more  engaged  and 
interactive  as  it  progressed.  For  instance: 

Leslie  (#13):  The  mentor  is  also  the  person  that  could  give  the  DE  student  that  social  benefit. 
[PHIII/A]  If  the  mentor  acts  as  an  advocate  for  the  student,  perhaps  gets  to  know  them  more  on  a 
personal  level,  then  the  DE  student  might  be  more  apt  to  have  a good  attitude  about  the  class,  even 
when  the  class  work  get  rough.  [PHIII/D] 

But,  even  the  students  with  the  best  attitudes  can  lose  motivation  and  start  feeling  insecure. 

[PHII/A]  If  there  are  good  people  supporting  the  program  (teacher,  mentor,  counselor,  parent 
community),  the  students  will  have  a much  better  chance  of  a successful  learning  experience. 
[PHIII/D]  So,  in  many  ways  the  DE  program  is  only  as  good  as  the  people  that  support  it. 

| PI  [III  L| 

Barbara  (#15):  Tying  in  attitude  and  the  cultural  differences  we  have  been  discussing  I am 
wondering  if  some  cultures  would  do  better  with  attitude.  [PHIV/A,  B]  I am  thinking  specifically 
of  Asian  countries  where  the  martial  arts  are  taught.  I am  currently  taking  Taekwondo  and  positive 
attitude  and  perseverance  are  two  of  the  tenants  that  are  taught.  [PHIV/C] 

In  the  example  above,  we  see  how  a discussion  develops  from  Phase  III  (sometimes  iteratively,  from  Phase 
III  back  to  Phase  II)  when  Leslie  integrates  and  synthesizes  the  information  through  co-construction  of  knowledge  to 
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Phase  IV  when  Barbara  tests  the  proposed  synthesis  against  both  received  fact  as  shared  by  others  and  her  existing 
cognitive  schema  as  well  as  her  experience. 

It  was  interesting  to  find  out  that  although  there  was  no  teacher  presence,  discussions  moved  from  early 
phases  to  later  phases.  Therefore,  it  is  possible  that  meaningful,  critical,  and  reflective  discussions  can  occur  without 
the  presence  of  an  instructor.  This  indicates  that  computer-mediated  conferences  can  promote  social  construction  of 
knowledge  and  collaborative  learning  in  a constructivist  learning  context. 

Overall,  the  findings  showed  that  knowledge  construction  took  place  in  early  phases  of  interaction  analysis 
model  by  Gunawardena  et  al.  (1997).  Higher  cognitive  skills  and  analytical  thinking  abilities  such  as  metacognitive 
activities  were  observed  little  in  the  discussions.  There  are  number  of  reasons  for  little  presence  of  higher  phases  of 
knowledge  construction.  One  reason  is  that  although  the  primary  medium  for  the  course  is  discussion  forum,  no 
guidance  or  structure  was  established  for  discussions.  This  might  have  prevented  students  to  concentrate  on  a topic 
in  order  to  make  critical  reflections  and  provide  thoughtful  comments  in  a timely  manner.  The  recommendation  is 
for  instructors  to  structure  discussions  in  a way  that  students  will  have  a common  set  of  rules  for  discussions 
including  participation  requirements,  certain  dates  for  initial  posts,  certain  discussion  topics,  and  description  of  the 
length  and  the  quality  of  messages. 

Another  possible  reason  for  the  little  presence  of  higher  phases  of  knowledge  co-creation  is  because  of  the 
lack  of  a facilitator  or  a moderator.  Findings  revealed  that  students  developed  knowledge  and  critical  thinking  skills 
at  higher  levels  when  there  was  strong  model  of  a facilitator.  In  addition,  it  was  also  observed  from  interaction  maps 
that  the  meaningful  remarks  made  by  a facilitator  promoted  the  quality  and  quantity  of  interaction  among  students. 
The  second  recommended  instructional  strategy  for  instructors  is  to  be  an  active  member  in  discussion  and  play  a 
role  in  guiding  students  toward  higher  levels  of  learning.  According  to  Hara  et  al.  (1998),  the  discussion  moderator 
“is  a key  player  in  determining  the  depth  of  dialogue  and  overall  knowledge  generation  processes.”  (p.  28). 
Gunawardena  et  al.  (1997)  also  point  out  that  the  moderator  “is  open  to  conceptualizing  the  learning  process  as  joint 
construction  of  knowledge  and  negotiation  of  meaning.”  (p.  428). 

Conclusion 

Understanding  the  experience  of  learning  and  the  overall  pattern  of  knowledge  construction  that  emerges 
from  computer-mediated  conferencing  assists  educators  to  acquire  intended  and  worthwhile  learning  outcomes.  This 
study  explored  the  dynamics  of  learning  community  at  a graduate  level  course  facilitated  by  computer-mediated 
conference  in  terms  of  two  main  concern  areas:  interaction  patterns  and  knowledge  construction  through  social 
negotiation  among  students.  Using  Gunawardena  et  al.  model  (1997),  co-construction  of  knowledge  was  found 
primarily  at  the  first  phase  as  a result  of  students’  conversation.  On  the  other  hand,  several  exchanges  showed 
evidence  of  movement  from  Phase  I to  higher  phases.  The  findings  show  that  a lack  of  structured,  organized 
discussion  and  the  absence  of  a moderator  may  have  contributed  to  these  results. 

The  analysis  of  data  revealed  that  knowledge  construction  through  social  interaction  took  place.  Students 
built  on  others’  contributions  to  co-create  knowledge.  However,  this  learning  experience  could  be  optimized  to 
increase  the  participation  level  and  obtain  high  phases  of  Interaction  Analysis  Model  (Gunawardena  et  al.,  1997). 
One  of  the  students  expressed  a need  for  participation  in  discussion  forum  as  quoted  below, 

I think  it's  important  to  establish  a comfortable  forum  for  student  discussion.  For  me  to 
participate,  it's  important  that  1 feel  that:  a)  1 have  something  meaningful  to  contribute,  b)  I can 
post  those  reflections  in  a non-threatening  environment,  and  c)  my  reflections  serve  a meaningful 
purpose  (i.e.  they  contribute  to  my  learning  and  the  learning  of  others  in  the  class  and  are  not 
simply  posts  for  the  sake  of  having  us  post  to  the  discussion  board). 

Rich  instructional  systems  and  learning  experiences  can  be  designed  by  understanding  how  knowledge  is 
constructed  and  distributed  in  CMC  environment.  Further  research  is  needed  in  understanding  how  students  develop 
cognitively  demanding  knowledge  through  computer  conferences.  What  are  the  factors  and  pedagogical  strategies 
that  affect  higher-order  thinking  skills  and  knowledge  construction  in  a collaborative  online  learning  environment? 
Does  online  discussion  as  a component  of  a blended  learning  environment  promote  social  construction  of 
knowledge?  There  remain  many  questions  and  areas  to  be  further  investigated.  These  studies  will  help  us  understand 
the  richness  and  potential  of  computer  conferencing  in  learning  experience. 
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Abstract 

Accrediting  bodies,  such  as  the  National  Council  for  the  Accreditation  of  Teacher  Education  (NCATE), 
strive  to  improve  the  quality  of  teacher  preparation  programs.  Through  assessment-based  analysis  of  institutional 
practices,  NCATE  accredits  institutions  of  higher  education  that  successfully  address  six  Unit  Standards.  Although 
all  the  NCATE  Standards  are  important,  this  article  is  focused  on  aspects  and  issues  surrounding  Standard  2, 
Assessment  and  Unit  Evaluation.  It  is  the  intention  of  the  authors  that  their  experiences  with  the  NCATE 
accreditation  process  will  help  other  institutions  develop  their  own  system,  with  an  understanding  of  what  will  make 
it  a worthwhile  endeavor.  What  follows  is  a combination  of  guidelines  for  building  the  assessment  system,  lessons 
that  we  learned,  and  words  of  wisdom  about  the  overall  process  of  accreditation. 


In  the  current  educational  climate  of  the  “No  Child  Left  Behind”  Act  and  significantly  increased 
standardized  testing,  many  states  within  the  United  States  are  mandating  that  institutions  of  higher  education 
involved  in  the  preparation  of  future  teachers  be  accredited  by  national  or  state  recognized  accrediting  agencies. 
Some  of  these  accrediting  bodies  include  the  National  Council  for  the  Accreditation  of  Teacher  Education  (NCATE) 
and  the  New  York  State  Board  of  Regents  and  the  Commissioner  of  Education.  This  move  toward  accreditation  has 
necessitated  the  in-depth  examination  of  all  teacher  education  programs  and  education  faculty  professional 
development. 

Accrediting  bodies,  such  as  the  National  Council  for  the  Accreditation  of  Teacher  Education  (NCATE), 
strive  to  improve  the  quality  of  teacher  preparation  programs.  Through  assessment-based  analysis  of  institutional 
practices,  NCATE  accredits  institutions  of  higher  education  that  successfully  address  six  Unit  Standards,  which 
include:  1)  Candidate  Knowledge,  Skills,  and  Dispositions,  2)  Assessment  System  and  Unit  Evaluation,  3)  Field 
Experiences  and  Clinical  Practice,  4)  Diversity,  5)  Faculty  Qualifications,  Performance,  and  Development,  and  6) 
Unit  Governance  and  Resources.  In  addition,  the  various  teacher  education  programs  within  an  academic  institution 
must  also  meet  the  program  standards  within  their  respective  professional  associations.  The  process  requires 
comprehensive  curricular  reviews  to  determine  the  extent  of  compliance  with  the  individual  national  associations 
(National  Council  of  Social  Studies,  National  Council  of  Teachers  of  English,  National  Society  for  Teachers  of 
Mathematics,  National  Science  Teachers  Association,  for  example).  Each  organization  has  a series  of  standards  that 
define  the  goals  and  objectives  for  teachers  and  teacher  candidates.  Designing  or  redesigning  teacher  preparation 
programs  from  standards  subsequently  aligned  to  assessment  is  a vastly  different  undertaking  than  designing 
programs  from  a curricular  base,  which  is  what  has  traditionally  been  done  in  teacher  education  (Pankratz,  2004). 
Since  viewing  teacher  education  through  a standards  assessment  lens  rather  than  through  a curricular  lens  is  a 
somewhat  new  initiative,  it  can  prove  at  times  to  be  quite  challenging. 

To  better  understand  the  NCATE  accreditation  process,  it’s  important  to  understand  some  terminology  used 
by  the  agency.  As  accreditation  efforts  help  to  ensure  that  the  education  programs  within  an  institution  of  higher 
education  meet  the  needs  and  expectations  of  the  entire  professional  community,  it’s  important  to  define  that 
community  and  its  constituents.  According  to  the  NCATE  Glossary  of  Terms,  the  Professional  Community  is 
comprised  of  "full-  and  part-time  faculty  (including  clinical  faculty)  in  the  professional  education  unit,  faculty  in 
other  units  of  the  college/university,  P-12  practitioners,  candidates,  and  others  involved  in  professional  education" 
(NCATE,  2006).  Essentially,  the  community  includes  all  stakeholders  that  play  any  type  of  role  in  a teacher 
preparation  program. 
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Within  the  Professional  Community  lies  the  Professional  Education  Unit  (PEU).  The  Unit  is  defined  as 
"the  institution,  college,  school,  department,  or  other  administrative  body  with  the  responsibility  for  managing  or 
coordinating  all  programs  offered  for  the  initial  and  continuing  preparation  of  teachers  and  other  school  personnel, 
regardless  of  where  these  programs  are  administratively  housed"  (NCATE,  2006).  At  most  institutions,  the  PEU 
consists  of  faculty  from  a variety  of  areas  of  academic  study.  These  can  include  English  Language  Arts,  Foreign 
Language,  Mathematics,  Sciences,  Social  Studies,  Fine  Arts,  Business  and  Economics,  and  of  course,  programs 
within  education.  In  addition,  school  districts  also  play  an  important  role  in  preparing  teacher  candidates,  both  as 
affiliates  who  support  education  programs  and  as  consumers  who  rely  on  the  institution  as  a source  of  quality 
educators,  administrators,  teacher  resources,  and  professional  expertise.  The  most  essential  members  of  the 
Professional  Community  include  teacher  education  candidates,  who  interact  with  all  of  the  other  members  of  the 
Professional  Community,  influencing  and  being  influenced  by  all  of  the  previously  mentioned  groups. 

It  is  in  conjunction  with,  and  with  the  cooperation  of  these  aforementioned  groups,  that  institutions  of 
higher  education  attempt  to  address  the  six  Standards  outlined  by  NCATE.  Although  all  the  NCATE  Standards  are 
very  important,  this  article  is  focused  on  aspects  and  issues  surrounding  Standard  2,  Assessment  and  Unit 
Evaluation.  It  is  the  intention  of  the  authors  that  their  experiences  with  the  NCATE  accreditation  process  will  help 
other  institutions  develop  their  own  system,  with  an  understanding  of  what  will  make  it  a worthwhile  endeavor. 

What  follows  below  is  a combination  of  areas  in  which  we  were  successful,  lessons  that  we  learned,  and  words  of 
wisdom  along  the  lines  of  “if  we  knew  then  what  we  know  now.” 

First  and  foremost,  is  important  to  have  the  administration  at  your  institution  fully  supportive  of  the 
accreditation  effort.  In  general,  this  is  not  difficult  to  acquire  as  most  administrations  realize  the  criticality  of 
accrediting  their  colleges,  schools,  and/or  divisions  of  education.  (In  some  states,  accreditation  by  a federally  or 
state-recognized  accrediting  body  is  required!)  However,  while  verbal  support  from  administration  for  accreditation 
is  often  forthcoming,  it  is  important  that  money,  release  time,  and  most  importantly,  institutional  cooperation  also  be 
given. 

Monetary  support  is  essential  for  many  of  the  initiatives  that  need  to  be  undertaken  to  achieve  accreditation. 
It  is  likely  that  a dedicated  “NCATE  Coordinator”  will  need  to  be  hired  - or  at  least  given  release  time  - to  organize 
this  significant  undertaking.  Many  aspects  need  to  be  organized  for  the  accreditation  process,  including  data 
collection,  subcommittee  formation,  and  cooperation  from  other  areas  of  the  university  such  as  administration,  the 
arts  and  sciences,  and  media  services.  As  well,  writing  the  final  institutional  report  is  an  enormous  job  and  clearly 
needs  to  be  facilitated  by  one  individual  so  that  the  document,  likely  put  together  by  various  individuals  or  groups, 
reads  as  if  written  in  one  voice.  These  are  merely  a few  of  the  vast  responsibilities  of  an  NCATE  Coordinator  and 
this  individual  needs  to  be  compensated  by  the  institution  in  some  regard. 

Another  undertaking  that  requires  monetary  support  is  data  collection  for  unit  assessment  purposes  in  order 
to  provide  data,  or  evidence,  that  students  are  attaining  goals  or  objectives  supported  by  the  institution’s  conceptual 
framework.  Often  institutions  do  not  consider  the  cost  of  mass  mailings  of  instruments  such  as  alumni  or  employer 
surveys,  but  for  larger  institutions,  these  could  number  in  the  thousands  for  any  individual  survey.  As  well  as 
mailing  costs,  photocopying,  paper,  envelopes,  and  return  envelopes,  also  add  to  overall  costs.  Further,  once  these 
instalments  are  returned,  analysis  of  the  data  takes  place  and  in  one  form  or  another,  this  also  costs  money.  If  these 
instillments  are  electronic,  then  the  development  of  web-based  surveys  could  necessitate  purchasing  specialized 
software  or  paying  a programmer/developer.  If  survey  instruments  are  hardcopy,  then  data  analysis  needs  to  be 
done  by  hand.  While  a committee  chair  or  professor  could  do  this,  it  is  advisable  to  hire  a research  assistant  to 
complete  this  extensive,  and  often  times  tedious,  work.  As  this  type  of  data  collection  is  meant  to  be  a continuous, 
sustainable  endeavor  it  is  important  that  a position  or  part  of  an  individual’s  workload  be  dedicated  to  this  task  and 
supported  at  the  outset  of  undertaking  accreditation.  However,  it  is  strongly  advised  that  an  electronic  data- 
gathering/data-analysis  interface  be  adopted.  This  is  preferable  not  only  because  it  is  a more  efficient  way  to  work 
with  necessary  data,  but  it  also  facilitates  the  dissemination  of  gathered  data  to  the  involved  parties,  i.e.  program 
chairs,  directors  of  student  teaching,  deans,  etc.  It  is  key  that  these  various  groups  or  individuals  communicate 
easily  and  have  access  to  candidate  information  as  they  progress  through  the  various  teacher  preparation  programs. 

Other  monetary  expenses  may  come  in  the  form  of  course  release  or  extra  service  pay  for  faculty  or  staff. 
While  often  institutions  give  faculty  the  assurance  that  work  on  the  accreditation  effort  will  count  as  scholarship  or 
service  work,  to  the  rest  of  the  professional  communities,  that  doesn’t  count  for  much  in  terms  of  an  individual’s 
professional  profile.  As  well,  accreditation  can  be  the  whole  of  an  individual’s  service  work  and  then  some.  It  is 
important  that  faculty  working  on  the  accreditation  effort  sacrifice  neither  their  scholarly  activities,  nor  the  entirety 
of  other  service  work,  to  accomplish  this  monumental  task.  It  is  therefore  important  that  they  be  given  time  or  other 
compensation  to  complete  work  associated  with  NCATE  accreditation. 
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The  writing  of  the  institutional  report  is  a difficult  endeavor  at  best  and  involves  the  cooperation  and 
collaboration  of  many  diverse  parties  and  individuals.  It  takes  an  individual  who  has  a certain  amount  of  seniority 
within  an  academic  institution,  as  well  as  one  who  has  substantial  experience  with  the  NCATE  process.  It’s  best  if 
the  individual  coordinating  the  writing  of  the  report  is  the  NCATE  coordinator  and/or  senior  faculty.  If  this 
individual  prefers  to  pass  the  writing  or  editing  of  the  Institutional  Report  on  to  a faculty  member  who  may  be  a 
better  writer,  that  can  work  also,  however  the  organizing  and  managing  of  the  report  needs  to  remain  in  the  hands  of 
the  NCATE  Coordinator.  The  reason  for  this  is  that  the  information  for  many  aspects  of  the  report  need  to  be  culled 
from  administration,  institution-wide  committees,  personnel  in  charge  of  university  resources  and  governance, 
faculty  in  the  arts  and  sciences,  as  well  as  faculties  and  committees  within  education.  Within  normal  institutional 
operations  it  is  quite  difficult  to  coordinate  all  these  sources  of  information,  not  to  mention  trying  to  do  so  under  a 
deadline.  Only  an  individual  with  some  clout  in  the  university  could  feasibly  do  this  with  any  success  and 
expediency.  Not  to  mention  that  only  a long-standing  member  at  an  institution  would  know  where  to  go  for  needed 
information  without  having  to  spend  extra  time  researching  such  resources.  Clearly,  it’s  unadvisable  to  leave 
accreditation  solely  to  junior  faculty,  although  certainly  they  can  play  important  and  necessary  roles  within  such  an 
effort. 

To  that  end,  it  is  important  that  the  whole  or  the  majority  of  accreditation  responsibilities  not  fall  to 
untenured  faculty.  Unfortunately,  more  often  than  not,  untenured  faculty  are  the  ones  who  are  given  various 
accreditation  jobs  to  complete  without  having  the  proper  knowledge  or  support  to  accomplish  them.  It  is  senior 
(tenured)  faculty  who  should  be  working  on  the  various  accreditation  initiatives  and  reports,  as  they  are  the  ones 
who  are  able  to  most  efficiently  and  effectively  complete  this  work.  Senior  faculty  has  knowledge  of  the 
institution’s  resources,  human  or  otherwise,  to  enable  them  to  implement  necessary  initiatives  or  acquire  needed 
information  quickly.  Specifically,  facilitating  the  acquisition  of  assessment  data,  the  discussion  with  the  involved 
faculties  of  data-driven  conclusions  that  may  modify  professional  practice,  and  most  importantly,  the  writing  of  the 
institutional  report,  should  all  be  undertaken  by  dedicated,  tenured  faculty  members. 

Before  you  enter  into  the  accreditation  process,  it  is  important  to  remember  not  only  the  items  discussed 
above,  but  also  the  following  set  of  guidelines.  These  guidelines  will  assist  in  creating  an  overall  approach  to 
accreditation  with  not  only  an  eye  on  the  beginning  stages,  but  also  the  continuing  processes,  sustainable  initiatives, 
and  final  outcomes.  They  facilitate  an  overall  view  of  accreditation  in  more  or  less  a linear  fashion,  and  assist  in 
making  it  a much  more  effective  and  efficient  process.  The  recommended  guidelines  for  NCATE  accreditation  are 
as  follows: 

1 . Create  your  conceptual  framework  with  an  eye  heavily  trained  on  performance -based  outcomes  and 
assessment.  Make  it  a collaborative,  group  effort,  involving  members  from  all  areas  of  the  professional 
unit  (NCATE,  2000).  Critical  to  the  accreditation  process  when  defining  the  framework  is  the  involvement 
of  faculty  from  the  Arts  and  Sciences.  Input  from  all  these  groups  and  individuals  will  ensure  a well- 
rounded  and  representative  work  including  philosophies  and  ideologies  from  all  professionals  involved 
with  teacher  candidate  preparation. 

2.  Define  your  objectives  from  your  conceptual  framework.  Make  them  clear  and  straightforward.  The  group 
that  created  the  conceptual  framework  is  generally  the  best  group  to  create  the  objectives. 

3.  Form  a different  committee  to  create  assessment  outcomes.  While  the  group  that  created  the  conceptual 
framework  and  resulting  objectives  can  also  create  the  assessment  outcomes,  it  is  better  for  a group  with  a 
fresh  perspective  to  do  this.  It  will  be  easier  for  them  to  align  the  assessment  with  the  objectives  without 
the  confounding  nuances  of  the  conversations  that  surrounded  the  creation  of  the  conceptual  framework 
and  objectives.  Be  sure  to  include  representatives  from  all  teacher  certification  programs,  field  placement 
and  Arts  and  Sciences.  It  is  also  crucial  to  the  assessment  effort  to  ask  that  a representative  from 
Institutional  Research  or  Alumni  Affairs  sit  on  the  committee  as  well.  This  is  important  because  the 
assessment  effort  is  larger  than  teacher  education  candidates  attending  university.  Data  collection  may  also 
extend  to  alumni,  employers,  field  supervisors,  parents,  P-12  students,  and  community  members. 

4.  Define  your  assessment  outcomes  from  your  conceptual  framework  objectives.  Make  them  performance- 
based  and  behavioral.  It’s  important  that  they  are  written  in  terms  that  are  measurable  rather  than  general 
or  vague.  This  is  essential  when  defining  dispositional  outcomes,  which  are  often  hard  to  identify.  It  is 
helpful  to  thoroughly  research  this  area  in  order  to  understand  the  difficulties  of  measuring  dispositions 
quantitatively. 

5.  Since  unit  assessment  involves  all  teacher  certification  programs,  it  is  important  to  bring  the  assessment 
outcomes  that  the  committee  created  to  all  faculty  members  involved  in  the  accreditation  effort.  This  can 
be  accomplished  either  through  assessment  committee  representatives  bringing  the  outcomes  to  their  home 
departments  or  having  a large  general  session  where  all  faculty  could  attend  and  the  outcomes  could  be 
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presented,  discussed,  modified,  and  hopefully,  adopted.  It  is  important  that  this  step  occur  because  NCATE 
accreditation  is  not  meant  to  be  simply  a “paper  chase.”  It  is  meant  to  be  a process  that  promotes  systemic 
change  within  teacher  education  and  preparation.  In  order  for  this  to  occur,  all  individuals  concerned  with 
teacher  preparation  and  accreditation  need  to  be  involved  in  decision-making,  either  in  a large  group  or 
small  committee.  This  allows  faculty  and  members  of  the  academic  unit  to  take  ownership  of  the 
accreditation  process  and  any  changes  resulting  from  it.  As  well,  when  all  are  involved  in  decision-making, 
each  individual  is  able  to  discuss,  and  thereby  promote,  the  goals,  frameworks,  outcomes,  etc.  that  are 
adopted  by  the  unit  at  large.  This  promotes  an  institutional  culture  in  which  constituents  are  informed  and 
invested  in  accreditation,  making  the  process  less  of  a set  of  hoops  to  jump  through  but  rather,  an 
opportunity  to  reflect  upon  and  improve  how  teacher  education  candidates  are  prepared. 

6.  It  is  also  important  not  to  forget  that  unit  assessment  involves  the  assessment  of  unit  operations  as  well  as 
assessment  of  degree-granting  programs.  Unit  operations  encompass  activities  such  as  the  evaluation  of 
advisement  and  student  teaching  placements.  It  also  involves  the  review  and  evaluation  of  the  assessment 
system  as  a whole  and  an  institution’s  conceptual  framework.  Ordinarily,  a university  or  college  has 
evaluation  measures  already  in  place  for  areas  like  academic  advisement  so  it  is  only  a matter  of  obtaining 
the  necessary  information  from  the  appropriate  office  and  documenting  the  process.  For  endeavors  such  as 
assessing  conceptual  frameworks  or  assessment  systems,  new  procedures  may  need  to  be  developed, 
implemented  and  documented. 

7.  Once  the  academic  unit  has  adopted  the  assessment  outcomes,  align  them  to  both  the  Special  Professional 
Association  (SPA)  standards  for  each  degree-granting  program.  In  this  way,  when  the  assessment 
outcomes  are  subsequently  aligned  to  existing  data  collection  instruments,  the  instruments  measure  both 
assessment  outcomes  and  SPA  standards.  Also,  it  is  more  than  likely  that  modification  of  the  instruments 
will  be  necessary  to  better  measure  the  outcomes.  In  addition,  some  of  the  outcomes  may  not  be  assessable 
with  the  current  instruments. 

8.  Create  instruments,  processes,  or  programs  to  measure  the  remainder  of  the  assessment  outcomes.  Be  sure 
to  have  a mixture  of  both  qualitative  and  quantitative  measures  for  the  overall  collection  of  outcomes. 

More  than  one  assessment  measure  for  any  one  outcome  is  desired,  but  not  required.  Be  sure  that  the 
assessment  measures  or  processes  that  are  put  into  place  are  sustainable,  reasonable,  and  manageable  in 
terms  of  an  overall  assessment  program.  If  unreasonable  assessment  procedures  are  established,  it  is  likely 
that  data  will  not  be  able  to  be  collected  over  time  and  therefore  candidate  change,  growth  and 
improvement  in  attaining  the  knowledge,  skills,  and  dispositions  put  forth  in  the  conceptual  framework  will 
be  impossible  to  track. 

9.  It  is  especially  crucial  to  have  input  from  members  of  the  PEU  from  P-12  environments  throughout  the 
assessment  process,  but  especially  in  steps  7 and  8.  It  is  with  their  help  that  manageable  instruments  will 
be  created  and  adopted,  and  ultimately  implemented.  If  the  P-12  constituents  are  a part  of  this  endeavor 
from  the  outset,  they  will  feel  invested  in  the  conceptual  framework,  assessment  system,  and  the 
accreditation  effort  as  a whole.  Without  their  cooperation,  much  needed  evidence  will  not  be  gathered  nor 
will  the  conceptual  framework  be  promoted  to  all  facets  of  education  candidates’  experiences. 

10.  Align  data  collection  instruments  and  evidence  to  the  Four  Transition  Points  outlined  by  NCATE.  It  is  fine 
to  have  more  transitions,  gateways,  or  checkpoints  than  those  that  are  reviewed  by  the  national  accrediting 
body,  but  it  is  important  that  there  are  not  less  than  four.  The  four  transition  points  NCATE  examines  are 
1)  entrance  into  education  programs;  2)  entry  into  student  teaching,  internship,  practicum;  3)  exit  from 
student  teaching,  internship,  practicum;  and  4)  exit  from  education  programs.  Divide  these  four  transition 
points  into  initial  and  advanced  certification  programs  as  both  types  of  certification  must  be  accounted  for 
at  each  point.  Be  clear  what  types  of  assessments  are  gathered  and  reviewed  at  each  transition  point. 

Create  rubrics  or  descriptors  of  acceptable  performances/scores/etc.  for  each  assessment  at  each  point. 

State  with  what  body,  office,  or  person,  the  transition  points  are  tracked  for  each  candidate  and  how 
successful  transition  from  one  point  to  the  next  is  communicated  to  necessary  individuals  (i.e.  program 
chairs,  unit  assessment  chairs,  deans,  etc.). 

11.  Investigate  and  adopt  efficient  data  collection  measures  and  procedures.  Create  electronic  databases  and 
programming  routines  to  extract  standardized  test  data  and  other  demographic  data  that  may  be  available 
for  assessment  use.  Be  sure  answers  to  your  surveys  can  be  compiled  and  analyzed  electronically.  Web- 
based  instillments  that  export  into  programs  such  as  Microsoft  Access,  Microsoft  Excel,  or  SPSS  are 
preferable.  As  well,  server  space  that  is  common  to  all  stakeholders  in  the  PEU  should  be  attained  to  house 
these  databases.  If  that  is  not  an  option,  create  surveys  that  are  answerable  in  Scantron  format  so  that  the 
hardcopies  can  simply  be  run  through  a machine  and  answers  aggregated  and  data  tabulated.  Using  entirely 
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hardcopy  data  collection  instruments  is  not  viable  for  sustained  data  collection.  A full-time  research 
assistant  would  need  to  be  hired  to  input  and  analyze  data  from  numerous  candidate  surveys,  employer 
surveys,  and/or  alumni  surveys.  This  is  especially  pertinent  if  you  have  large  population  of  pre-service 
teacher  candidates.  You  may  find,  initially,  that  your  institution  has  a multitude  of  data  already,  but 
collected  in  a variety  of  places  and  databases.  It  is  important  to  bring  this  information  together  in  one  place 
in  order  to  make  analyses  more  efficient  and  to  facilitate  the  communication  of  important  information.  The 
ability  to  link  all  of  the  data  collection  measures  and  databases  will  be  an  issue  that  should  be  addressed  at 
the  planning  stages  of  your  Assessment  System. 

12.  Prior  to  beginning  any  data  collection,  attain  approval  or  exemption  from  the  Use  of  Human  Participants 
examining  board  at  your  institution.  This  is  a critical  step  in  ethical  assessment  and  data  gathering 
processes.  To  utilize  all  the  data  that  you  will  likely  need  in  your  assessment,  such  as  GPAs,  curriculum 
codes,  perhaps  gender,  ethnicity  or  age,  you  will  need  to  attain  this  type  of  permission  from  your  university. 

13.  Begin  data  collection.  Be  specific  about  when  you  would  like  surveys  or  information  returned  from  your 
various  participant  populations.  Include  cover  letters  that  explain  the  accreditation  process  and  the 
importance  of  responses  with  surveys  to  potential  employers  or  alumni.  Be  sure  that  supervisors  or 
professional  development  educators  are  aware  of  candidate  deadlines  for  returning  data  collection 
instruments.  This  will  help  to  avoid  updating  “finalized”  data. 

14.  Input  and  analyze  incoming  data  as  soon  as  it  is  returned.  Procrastinating  on  this  effort  merely  leaves  piles 
paper  and  many  computer  files  lying  around  with  the  potential  for  getting  lost  or  misplaced.  It  is  also 
critical  to  analyze  data  immediately  so  that  timely  conclusions  and  decisions  about  improving  professional 
practice  can  be  made  and  implemented. 

15.  Be  sure  to  aggregate  and  disaggregate  incoming  data.  Aggregated  data  is  useful  for  assessment  committee, 
unit,  and  administrative  purposes.  Disaggregated  data  is  more  useful  to  individual  degree-granting 
programs  as  well  as  departments  in  the  arts  and  sciences  that  prepare  teacher  education  candidates  with 
specific  academic  concentrations.  Both  aggregated  and  disaggregated  data  must  be  efficiently  disseminated 
to  the  appropriate  programs,  governing  bodies,  and  offices  so  that  conclusions  can  be  drawn  about 
professional  preparation  of  teacher  education  candidates.  These  types  of  data  can  also  be  used  as  evidence 
during  an  NCATE  site  visit.  It  is  therefore,  imperative  that  all  data  collection  instillments  are  designed  so 
as  to  identify  a variety  of  anonymous,  demographic  information  (such  as  degree  program,  arts  and  science 
concentration,  year  in  university,  etc.).  These  demographic  data  will  also  assist  in  linking  all  gathered 
information  together  and  analyzing  it  for  trends  across  programs,  concentrations,  gender,  etc. 

16.  Once  data  collection  measures  have  been  are  in  place,  measures  of  fairness  and  consistency  should  be 
ensured.  If  standardized  tests  are  being  used,  reliability  and  validity  information  can  generally  be  obtained 
from  the  testing  companies.  When  “homegrown”  surveys  are  in  use,  true  statistical  validity  and  reliability 
testing  may  not  be  an  option.  It  is  exceedingly  difficult  to  validate  self-made  questionnaires  so  other 
measures  of  ensuring  fairness  and  consistency  should  be  implemented.  One  example  would  be  to  assemble 
a small  committee  of  individuals  not  involved  in  the  original  alignment  of  “homegrown”  instillments  to 
conceptual  framework  outcomes.  This  ad  hoc  committee  could  then  examine  each  of  the  instmments  and 
document  whether  they  do  or  do  not  align  to  certain  outcomes  based  on  specific  criteria.  Granted,  this  is 
not  a way  to  ensure  validity  or  reliability  in  the  statistical  sense,  but  it  is  taking  a step  toward  ensuring 
fairness  and  consistency  and  that  is  all  that  NCATE  requires.  (In  fact,  when  reviewing  NCATE 
documentation,  they  do  not  use  the  terms  “validity”  and  “reliability.”  Instead,  their  exact  language  is 
“fairness”  and  “consistency.”) 

1 7.  As  an  assessment  committee,  distribute  the  aggregated  and  disaggregated  data  to  chairs  of  the  education 
departments,  chairs  of  the  arts  and  sciences  departments  which  participate  in  teacher  education,  chairs  of 
the  NCATE  standards  committees  (if  your  institution  has  those),  members  of  a unit  governing  body  and 
other  involved  stakeholders.  This  can  take  a vast  amount  of  time  to  organize  initially,  but  with  the  help  of  a 
graduate  assistant  or  work-study  student,  the  process  can  progress  much  more  quickly.  It  is  also  helpful  to 
include  a detailed,  explanatory  cover  letter  with  your  data,  explaining  the  instmment(s)  from  which  is  was 
obtained,  the  purpose  for  obtaining  the  data,  and  how  it  was  analyzed.  Once  the  data  is  properly  organized 
and  the  cover  letter  created,  it  is  simply  a matter  of  making  photocopies  and  doing  (mostly)  inter-office 
mailings. 

18.  Allow  some  time  for  departments  and  committees  to  review  the  aggregated  and  disaggregated  data  and  to 
draw  conclusions  based  on  the  data  analysis.  What  does  it  indicate  about  the  professional  practice  within  in 
the  programs?  Do  areas  need  to  be  modified?  Do  certain  aspects  need  to  be  recognized  and  praised  for 
their  success?  Questions  such  as  these  should  be  carefully  considered  and  discussed. 
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19.  Once  smaller  groups  have  had  a chance  to  review  the  data,  each  should  decide  upon  recommendations  to 
make  to  the  overarching  unit-governing  group  regarding  modifications  to  teacher  candidate  preparation. 
Since  professional  practice  is  being  assessed  as  a “unit,”  the  recommendations  made  by  one  group  or 
committee  effect  all  the  education  programs  and  must  be  discussed,  decided  upon,  and  adopted  as  a whole. 
If  not,  it  is  not  “unit”  assessment. 

20.  If  the  larger  unit-governing  group  accepts  recommendations  for  the  modification  of  teacher  preparation  by 
program,  or  as  a whole,  then  the  revisions  must  follow  the  normal  institution  process,  or  “lines  of 
authority,”  for  programmatic  changes.  This  usually  involves  the  completion  of  standard  institutional  forms, 
acceptance  by  various  curriculum  committees,  university  governing  bodies,  and  ultimately,  the  signature  of 
the  university  or  college  president  and/or  provost. 

21.  Recommendations  to  modify  professional  teacher  educator  preparation  need  to  be  put  into  practice  in  a 
timely  manner.  These  recommendations  can  include  omitting  specific  application  procedures  because  they 
are  redundant,  raising  or  lowering  required  GPAs  or  standardized  test  scores,  or  emphasizing  the  use  of 
scaffolding  in  teaching  throughout  the  teacher  preparation  programs.  Modifications  can  be  of  a minor, 
more  administrative  nature,  or  more  far-reaching  into  education  courses  and/or  field  experiences. 

Whichever  the  case,  it  is  important  to  discuss  modifications  to  teacher  preparation  as  a large  group  and 
come  to  mutually  agreeable  conclusions.  Otherwise,  modifications  or  changes  in  practice  will  only  be  done 
by  a few,  rather  than  reaching  across  the  unit.  As  well,  a timeline  for  larger  changes  to  be  implemented 
may  be  needed  so  that  involved  individuals  can  work  on  and  institute  initiatives  in  a manageable  way. 

22.  Assess  modifications  using  reliable  and  valid  data  gathering  techniques  (surveys,  focus  groups,  pre-tests 
and  posttests  with  a spectrum  of  stakeholders).  In  some  cases,  be  sure  to  allow  enough  time  for  a new 
initiative  or  program  modification  to  take  hold  in  the  institution  before  assessing  it.  Assessing  a 
modification  too  early  can  result  in  skewed  data  and  an  inaccurate  perception  of  the  success  or  failure  of 
the  change.  Similarly,  assessment  of  a modification  too  late  can  result  in  the  propagation  of  an 
unsuccessful  or  even  harmful  change  much  too  long.  The  nature  of  the  program  modification,  as  well  as 
input  from  initiative  implementers  and  candidates,  will  likely  dictate  when  to  assess  a change  to  the 
professional  preparation  of  teacher  education  candidates. 

23.  Continue  to  communicate  data-driven  recommendations  and  improve  the  professional  preparation  of 
teacher  education  candidates. 

24.  At  the  time  of  this  writing,  for  institutions  having  NCATE  accreditation  visits  in  the  Fall  of  2003  or  later,  at 
least  three  years  of  data  or  evidence  of  candidates  attaining  goals  outlined  in  the  conceptual  framework  is 
required.  It  is  also  worth  noting  that  institutions  having  their  site  visit  at  this  time  or  later  need  to  have  their 
assessment  systems  in  place  and  operational.  No  longer  is  an  “assessment  plan”  acceptable  for 
accreditation.  Now  any  plans  must  be  put  into  place  and  evidence  of  its  successful  operation  must  be 
exhibited. 

25.  Once  an  assessment  system  has  been  established  and  is  in  use,  and  other  areas  of  the  NCATE  Standards  are 
also  progressing,  schedule  a “mock  visit”  with  a consultant  who  is  well  versed  in  the  process  of  NCATE 
accreditation.  The  sooner  before  an  actual  on-site  visit  that  this  “mock  visit”  occurs,  the  better.  It  is  likely 
that,  no  matter  how  hard  an  institution  has  worked  at  the  Standards,  the  consultant  will  find  areas  that  need 
improvements  and  modifications.  The  more  time  given  an  institution  to  make  these  required  changes,  the 
better  for  all  concerned.  Making  modifications  to  the  Standards,  especially  the  Assessment  System,  under 
a tight  time  frame  is  at  best  difficult  and  at  others,  not  possible  at  all.  A “mock  visit”  scheduled  at  the 
earliest  time  an  institution  is  ready  is  best. 

Although  accreditation  should  be  a positive  process  meant  to  reflect  and  improve  upon  the  professional 
practice  of  teacher-education  institutions,  the  process  often  evolves  into  something  quite  the  opposite.  Rather  than 
building  collegiality  and  creating  a collaborative,  efficiently  functioning  unit,  the  process  of  accreditation  can  seem 
like  “just  one  more  thing”  that  is  often  pushed  onto  the  shoulders  of  the  unfortunate  (or  untenured! ).  If  you  see 
emotions,  tempers,  and  stress  levels  running  high  the  closer  it  gets  to  your  site  visit,  be  assured  that  this  happens  at 
many  institutions.  The  important  thing  to  remember  is  that,  although  accreditation  is  a serious  process,  it’s  best  not 
to  take  yourself  too  seriously  as  you  go  about  getting  ready  for  a site  visit.  Many  jobs  and/or  duties  associated  with 
accreditation  are  equally  difficult  and  time  consuming.  It’s  important  to  remember  that  you’re  not  “the  only  one” 
and  to  be  gentle  with  both  yourself  and  your  colleagues  when  it  comes  to  deadlines,  work  to  be  done,  and  scouting 
about  for  necessary  information.  As  natural  as  it  is  to  feel  the  stress  of  this  process,  take  care  to  give  compliments 
and  support  to  your  colleagues.  It  will  not  only  improve  the  overall  atmosphere  of  your  unit  (which  is  quite  critical 
during  an  accreditation  visit),  but  it  will  also  help  to  keep  the  process  in  perspective  and,  hopefully,  to  feel  the  value 
of  what  it  is  that  you  and  your  colleagues  are  doing. 
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Finally,  do  not  forget  your  candidates.  The  reality  of  the  situation  is  that  this  process  is  for  the  betterment 
of  their  professional  preparation.  They  too  are  stakeholders  will  be  learning  about  the  process.  It  is  important  to 
keep  open  dialogue  with  these  members  of  the  educational  community  and  when  doing  so,  to  keep  a record  of  any 
discussions  because  they  add  to  the  assessment  and  the  overall  improvement  of  degree  programs. 

As  we  stated  in  the  beginning  of  this  article,  accreditation  is  a process  that  is  meant  to  improve  the  quality 
of  teacher  preparation  programs.  This  is  achieved  through  cooperation,  collaboration,  reflective  and  thoughtful 
practice,  and  performance-based  assessment  measures  that  drive  improvements  in  current  practice.  Periodically,  this 
can  seem  to  be  a prolonged  and  stressful  activity,  but  given  enough  time  to  engage  in  the  process  effectively, 
accreditation  can  be  a very  positive  activity.  It  is  also  often  helpful  to  listen  to  the  voices  of  those  who  have 
previously  been  through  and  learned  from  this  process.  Hopefully  this  article  has  value  and  helps  to  improve  the 
quality  to  teacher  preparation  programs  by  facilitating  the  accreditation  effort  for  institutions  of  higher  learning. 
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Aggregated  Approaches  to  Identifying  Community  and  its  Constituent 
Elements  in  Formal  Blended  Learning  Environments 

Richard  A.  Schwier 
Ben  K.  Daniel 
University  of  Saskatchewan 

Abstract 

This  paper  describes  a variety  of  methods,  techniques  and  procedures  combined  to  determine 
whether  an  online  learning  community  exists,  to  isolate  its  constituent  characteristics  and  understand 
interactions  among  them,  and  build  a dynamic  model  of  an  online  learning  community.  It  presents  analysis 
of  data  from  a three-year  research  program  on  virtual  learning  communities,  including  user  perceptions  of 
community  (sense  of  community  indices);  interaction  analysis  (density,  intensity,  reciprocity)  content 
analysis  (transcripts,  interviews,  and  focus  groups),  paired-comparison  analysis  (Thurstone  scaling)  and 
community  modeling  techniques  (Bayesian  Belief  Network  analysis). 


Introduction 

This  paper  describes  a set  of  approaches  used  to  measure  and  understand  the  characteristics  of  community 
discussed  in  greater  detail  in  Schwier  and  Daniel  (2007).  The  categories  of  analysis  included  identifying  a sense  of 
community,  isolating  characteristics  of  community,  comparing  characteristics  of  community,  and  building  a 
dynamic  Bayesian  Belief  Network  Model  of  a community.  The  procedures  of  the  analysis  involved  eliciting 
summative  judgments  by  participants,  identifying  variables  constituting  a virtual  learning  community  and  making 
paired  comparison  to  determine  the  relative  importance  of  the  various  characteristics.  Finally,  we  used  the  data  to 
construct  a computational  model  from  the  data — one  that  not  only  represents  the  interrelationships  among  variables, 
but  that  can  also  be  used  to  project  the  effect  on  the  community  when  its  constituent  elements'  values  are  changed  as 
a results  of  new  evidence.  This  paper  presents  in  the  general  the  key  results  of  the  studies  undertaken.  For  detailed 
description  of  the  studies  and  methods  employed  and  the  findings  see  Schwier  and  Daniel  (2007). 


Research  Context 

Our  analyses  draw  on  data  generated  over  three  years  of  online  communication  among  groups  of  graduate 
students  in  Educational  Communications  and  Technology  as  they  participated  in  seminars  on  the  foundations  of 
educational  technology  and  instructional  design.  Each  offering  of  the  classes  spanned  an  entire  semester  or  academic 
year.  The  classes  were  small  graduate  seminars  with  enrolments  from  six  to  thirteen  students,  and  each  class  met 
primarily  online,  but  with  monthly  group  meetings.  While  most  students  were  able  to  attend  the  group  meetings 
regularly,  every  class  cohort  had  members  who  participated  exclusively  or  mostly  from  a distance.  Given  the 
blended  nature  of  all  of  the  classes,  we  confine  our  conclusions  to  similar  environments,  and  emphasize  that  these 
results  cannot  be  generalized  to  environments  that  are  entirely  online,  or  entirely  face-to-face.  Flowever,  the  methods 
employed  in  the  studies  can  be  extended  to  study  similar  learning  environments.  Measures  employed  to  determine 
whether  a community  existed  among  the  students  include;  content  analysis,  focus  groups,  and  inteiviews  to  ascertain 
sense  of  a community  index  and  intensity  of  interaction;  paired  comparison  using  Thurstone  scaling  to  determine 
relative  importance  of  the  identified  characteristics  of  virtual  learning  communities;  and  Bayesian  Belief  Network 
for  building  a dynamic  computational  model  and  evidence -based  scenarios  to  query  and  update  the  model. 


Sense  of  Community  Indices 

In  gross  sense,  to  determine  whether  a community  existed,  we  employed  the  "Sense  of  Community  Index 
(SCI)"  (Chavis,  undated),  a classic  instalment  employed  broadly  in  the  field  of  community  psychology  (Chavis  & 
Wandersman  1990;  Chipuer  & Pretty,  1999;  Obst  & White,  2004).  The  Sense  of  Community  Index  (SCI)  measures 
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an  individual's  psychological  sense  of  community.  We  administered  the  SCI  at  the  beginning  and  end  of  a year-long 
course,  and  ran  a simple  T-test  on  the  data  to  see  if  there  was  any  change  in  measures  of  the  group's  sense  of 
community  by  the  end  of  the  course.  The  T-test  results  suggested  a significant  positive  growth  in  the  sense  of 
community  scores  from  the  beginning  to  the  end  of  the  course  (p<  .01).  Though,  the  reliability  of  SCI  is  often 
questionable,  despite  its  long  use,  we  anticipated  using  the  Classroom  Community  Scale  (CCS)  proposed  by  Rovai 
and  Jordan  (2004).  The  classroom  community  scale  is  similar  in  format  and  intent  to  the  SCI,  but  it  boasts  a higher 
reliability  estimate  for  the  full  scale  (Chronbach's  alpha  = .93)  and  the  subscales  (connectedness  = .92;  learning  — 
.87),  partially  attributable  to  the  higher  number  of  items  on  the  scale. 


Density  and  Intensity  of  Interaction 

Fahy,  Crawford  and  Ally  (2001)  proposed  several  useful  measures  of  describing  interaction  collectively  called 
Transcript  Analysis  Tool  (TAT).  The  TAT  includes  methods  of  measuring  density,  intensity  and  persistence  of 
interactions  in  transcripts  of  online  discussions.  We  drew  on  their  recommendations  and  extended  some  of  them  to 
analyze  interactions  in  our  data,  particularly  transcripts  of  asynchronous  discussions.  We  explored  how  individuals 
in  the  class  were  connected  to  each  other  using  density  as  measurement  proxy.  Fahy,  Crawford  and  Ally's  (2001) 
defined  density  as  "the  ratio  of  the  actual  number  of  connections  observed,  to  the  total  potential  number  of  possible 
connections."  Given  by  the  formula:  Density  = 2a/N  (N-l),  where  "a"  is  the  number  of  observed  interactions 
between  participants,  and  "N"  is  the  total  number  of  participants.  Density  is  a measure  of  how  connected 
individuals  are  to  others  in  a group,  and  the  idea  is  that  a higher  degree  of  connection  is  a positive  indicator  of 
community. 

For  our  own  calculations,  we  included  only  peripheral  (voluntary  or  additional)  communications  between 
people  by  eliminating  all  instances  of  required  postings  and  responses.  We  felt  that  peripheral  interaction  would 
provide  a stronger  measure  of  community,  given  that  required  communications  among  students  might  inflate  the 
actual  density  value.  In  the  case  of  one  of  our  groups,  we  discovered  a density  ratio  of  .78,  suggesting  that  78%  of 
the  possible  connections  were  made.  Density  = 2(122)/13(12)  = .782.  Although  there  are  no  baseline  data  to  make 
judgments  about  the  existence  of  community,  this  level  of  density  did  seem  to  suggest  a strong  level  of  connection 
among  participants. 

Fahy,  Crawford  and  Ally  also  recommend  considering  measures  of  intensity  to  determine  whether  participants 
are  authentically  engaged  with  each  other,  not  merely  carrying  out  their  responsibilities  in  a course.  They  argue  that 
it  is  a useful  measure  of  involvement  because  it  involves  measures  of  persistence  and  dedication  to  being  connected 
to  others  in  the  group.  One  measure  of  intensity  is  "levels  of  participation,"  or  the  degree  to  which  the  number  of 
postings  observed  in  a group  exceed  the  number  of  required  postings.  In  this  case,  students  were  required  to  make 
490  postings  as  part  of  the  course  requirements,  and  they  actually  made  858  postings,  yielding  a level  of 
participation  ratio  of  1.75.  Another  important  measure  for  the  purpose  of  understanding  community  was  Fahy, 
Crawford  and  Ally's  "S-R  ratio",  a formula  to  measure  the  parity  of  communication  among  participants.  We  referred 
to  this  as  a measure  of  "reciprocity",  and  we  felt  that  truly  engaged  groups  who  form  communities  will  exhibit  high 
degrees  of  reciprocity.  Details  of  our  analysis  are  reported  in  Schwier  and  Daniel  (2007). 


Characteristics  of  a Model  of  a Virtual  Learning  Community 

Building  on  previous  model  of  virtual  learning  community  (see  Schwier,  2001),  we  had  identified  fourteen 
characteristics  of  community  that  grew  out  of  the  theoretical  model,  from  the  analysis  of  interactions  among 
participants,  from  a content  analysis  of  transcripts  of  communication  among  community  participants,  and  from 
interviews  and  focus  groups.  While  the  process  to  this  point  was  disciplined  at  each  step,  the  intention  was  to  draw 
out  characteristics  that  might  be  important  in  formal  virtual  learning  communities;  the  purpose  was  not  to  validate  or 
compare  the  relative  significance  of  any  of  the  characteristics.  The  next  step  in  the  process  was  to  try  to  determine 
the  relative  importance  of  the  characteristics  that  were  drawn  from  these  various  sources.  We  had  a good  sense  of 
what  were  many  of  the  characteristics  that  comprised  the  communities  we  observed,  but  we  did  not  have  any  reliable 
information  about  which  characteristics  were  important,  which  were  trivial,  and  which  might  be  more  important  than 
others.  Figure  1 presents  the  characteristics  observed  for  the  definitions  of  the  variables  (see  Schwier  & Daniel, 
2007) 
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Figure  1.  Characteristics  of  a virtual  learning  community  (Schwier,  in  press) 

To  address  this  question,  we  developed  a paired-comparison  treatment  that  asked  participants  to  compare  each 
characteristic  of  a VLC  to  every  other  characteristic  and  choose  the  characteristic  they  believed  was  more  important 
to  the  community.  Twenty-three  students  who  had  completed  their  coursework  volunteered  to  participate  in  the 
study.  The  fourteen  characteristics  were  compared  against  each  other,  resulting  in  91  paired-comparisons  in  the 
treatment.  Authorware  Professional™  was  used  to  develop  the  treatment,  and  the  treatment  was  administered  on 
Windows-based  PC  workstations.  In  the  design  of  the  treatment,  care  was  taken  to  avoid  response  bias  and 
contamination  from  fatigue  by  presenting  each  pair  in  random  order  and  by  alternating  the  upper-lower  orientation 
of  each  characteristic  in  relation  to  the  characteristic  against  which  it  was  being  compared.  After  completing  the 
comparisons,  participants  were  asked  to  describe  how  they  made  their  decisions  generally,  and  if  there  were  factors 
that  influenced  their  decisions. 


Thurstone  Analysis 

In  order  to  discriminate  among  the  variables,  Schwier  & Daniel  (2007)  developed  a paired-comparison 
treatment  that  required  participants  to  compare  each  characteristic  of  a VLC  to  every  other  characteristic  and  choose 
the  characteristic  they  believed  was  more  important  to  the  community.  This  was  based  on  Thurstone's  method  of 
paired  comparisons,  a method  of  analysis  that  generates  a scale  ranking  and  scale  points  among  variables  that  can  be 
used  to  plot  a visual  representation  of  distances  between  and  among  variables. 

Thurstone  (1927)  postulated  that  for  each  of  the  items  being  compared  and  among  all  subjects,  a preference 
will  exist,  and  that  for  each  item  the  preference  will  be  distributed  normally  around  that  item's  most  frequent  or 
modal  response.  A person's  preference  for  each  item  versus  every  other  item  is  obtained,  and  the  more  people  that 
select  one  item  of  a pair  over  the  other  item,  the  greater  the  preference  for,  or  perceived  importance  of,  that  item,  and 
thus  the  greater  its  scale  weight.  Thurstone's  Law  of  Comparative  Judgment  circumvents  potential  ceiling  effect 
problems  by  forcing  individuals  to  rank  items  two  at  a time  rather  than  all  at  once  (Manitoba  Centre  for  Health 
Policy,  2005).  Given  the  results  of  all  possible  paired  comparisons  of  the  variables  under  study,  scale  values  can  be 
plotted  on  a line  to  provide  a graphic  illustration  of  the  relative  value  of  each  variable,  represented  by  its  relative 
distance  from  the  other  variables  (the  greater  the  distance  between  any  two  variables  on  the  scale,  the  greater  the 
differences  between  those  two  variables). 

The  scale  is  descriptive,  and  there  are  no  post-hoc  tests  available  to  identify  significant  differences  among 
variables.  But  the  scale  values  provide  a convenient  metric  for  assigning  initial  weights  to  variables  in  modeling 
exercises.  In  the  study  of  the  fundamental  variables  of  virtual  learning  communities,  Schwier  and  Daniel  (2007), 
compared  each  VLC  characteristic  with  the  others,  following  procedures  outlined  by  Misanchuk  (1988).  The  data 
were  then  converted  into  a line  drawing  that  depicted  differences  between  elements  along  a line.  Greater  differences 
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were  shown  spatially  as  larger  distances  between  points  on  the  line.  The  outcome  of  the  comparison  and  the  ranking 
of  the  variables  are  shown  in  Table  1. 

Table  1.  Thurstone  Scale  Rankings  and  Scale  Points  for  Each  of  the  Fourteen  VLC  Characteristics. 


Characteristic 

Thurstone  Scale  Ranking 

Thurstone  Scale  Point 

Trust 

1 

0.7341 

Learning 

2 

0.5806 

Participation 

3 

0.3182 

Mutuality 

4 

0.2671 

Intensity 

5 

0.2425 

Social  Protocols 

6 

0.1852 

Reflection 

7 

0.1523 

Autonomy 

8 

0.0155 

Awareness 

9 

-0.0785 

Identity 

10 

-0.1939 

Future 

11 

-0.2474 

Technology 

12 

-0.5033 

Historicity 

13 

-0.7309 

Plurality 

14 

-0.7701 

As  a result  of  this  analysis,  we  were  able  to  obtain  measures  that  could  be  used  to  understand  the  association 
and  interplay  of  community  characteristics  in  a VLC,  and  we  could  also  use  the  Thurstone  Scale  points  to  assign 
weights  to  these  characteristics  when  we  attempted  to  construct  a dynamic  model  of  virtual  learning  communities. 
Reviewing  the  results,  it  is  apparent  that  there  are  at  least  three  clusters  of  characteristics.  Trust  and  learning  were 
considered  by  the  participants  to  be  the  most  important  characteristics  of  a VLC.  A large  cluster  of  characteristics 
gathered  around  the  mean  scale  point,  and  while  they  differed  from  each  other,  we  treated  them  as  a group  because 
of  their  central  position  relative  to  the  other  points.  Technology,  historicity  and  plurality  were  ascribed  much  lower 
status  than  the  other  characteristics,  and  one  might  argue  as  a result  that  they  should  be  eliminated  from  the  model 
entirely.  After  reviewing  comments,  it  was  apparent  that  even  those  characteristics  that  were  positioned  at  the  low 
end  of  the  Thurstone  scale  still  had  a role  to  play  in  the  construction  of  community,  however  marginal  that  influence 
might  be. 

We  were  also  reluctant  to  eliminate  characteristics  at  this  point  in  the  research  because  we  are  still  gathering 
primary  data  from  new  groups.  Our  confidence  in  the  relative  positions  of  these  characteristics,  and  ultimately  our 
judgments  about  their  inclusion  in  a model  of  VLC,  will  grow  as  our  analysis  continues.  At  what  point  will  we  be 
satisfied  that  we've  identified  the  important  characteristics  and  measured  their  relative  importance?  Probably  never, 
given  that  VLCs  are  dynamic  environments  that  are  also  situated  in  particular  learning  contexts.  But  we  will 
continue  to  gather  data  to  develop  and  refine  models,  and  our  tools  and  the  sophistication  of  our  observations  will 
mature  over  time  too. 


Building  Models  of  Online  Learning  Communities 

Bayesian  networks,  Bayesian  models  or  Bayesian  belief  networks  (BBNs)  can  be  classified  as  part  of  the 
probabilistic  graphical  model  family.  Graphical  models  provide  an  elegant  and  mathematically  sound  approach  to 
represent  uncertainty.  BBN  approach  combines  advances  in  graph  theory  and  probability.  BBNs  are  graphs 
composed  of  nodes  and  directional  arrows  (Pearl  1988).  Nodes  in  BBNs  represent  variables  and  directed  edges 
(arrows)  between  pairs  of  nodes  indicate  relationships  between  variables.  The  nodes  in  a BBN  are  usually  drawn  as 
circles  or  ovals.  Further,  BBNs  offer  a mathematically  rigorous  way  to  model  a complex  environment  that  is 
flexible,  able  to  mature  as  knowledge  about  the  system  grows,  and  computationally  efficient  (Druzdzel  & Gaag, 
2000;  Rusell  & Norvig,  1995). 

Bayesian  networks  can  be  used  to  model  probabilistic  relationships  among  variables.  In  some  cases,  their 
graphical  structure  can  be  loosely  interpreted  as  the  result  of  direct  causal  dependencies  between  variables.  In 
domains  with  many  causal  relations,  such  as  in  medical  diagnosis  (symptoms  cause  diseases),  human  experts  are 
usually  able  to  express  their  domain  knowledge  in  the  graphical  structure  of  the  network.  For  example,  in  a model 
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for  medical  diagnosis,  the  parameters  of  the  network  are  the  conditional  probabilities  of  effects  given  the  state  of 
their  direct  causes. 

Bayesian  Belief  Network  (BBN)  techniques  are  increasingly  being  used  for  understanding  and  simulating 
computational  models  of  complex  social  systems.  BBN  models  enable  reasoning  when  there  is  uncertainty  (Pearl, 
1998).  They  combine  the  advantages  of  an  intuitive  visual  representation  with  a sound  mathematical  basis  in 
Bayesian  probability.  The  motivation  to  build  the  Bayesian  model  of  a virtual  learning  community  is  to  be  able  to 
perform  a number  of  simulations  and  observe  the  influence  of  variables  in  the  model  with  the  goal  of  determining 
and  understanding  the  relationships  among  these  variables  that  are  critical  to  learning  in  communities. 

The  first  step  in  building  a BBN  is  to  identify  key  variables  that  represent  a domain  (Druzdzel  & Gaag,  2000; 
Pearl,  1988;  Rusell  & Norvig,  1995).  The  variables  identified  in  our  model  are  drawn  from  an  analysis  of  online 
transcripts,  interviews  and  email  traffic  that  were  subjected  to  grounded  theory  analysis,  and  the  identified  variables 
were  then  subjected  to  a Thurstone  analysis  to  identify  their  relative  weights.  The  motivation  to  build  the  Bayesian 
model  of  a virtual  learning  community  is  to  be  able  to  perform  a number  of  simulations  and  observe  the  influence  of 
variables  in  the  network  with  the  goal  of  determining  and  understanding  those  variables  that  are  critical  to  virtual 
learning  communities  as  well  as  their  interactions  in  the  processes  of  learning.  In  building  the  model,  once  variables 
were  identified,  the  second  step  involved  mapping  the  variables  into  a graph  (see  Figure  2)  based  upon  coherent 
qualitative  reasoning. 

Druzdzel  and  Henrion  (1993)  proposed  a transformation  of  a causal  Bayesian  network  into  a qualitative 
probabilistic  network  (QPN),  in  which  the  relation  between  two  adjacent  nodes  is  denoted  as  positive  (+),  negative  (-), 
null  (0)  or  unknown  (?);  there  are  also  relations  that  involve  more  than  two  nodes,  such  as  positive  or  negative  synergies. 
The  main  advantage  of  QPN's  is  that  they  simplify  the  construction  of  models,  because  they  do  not  require  the  elicitation 
of  numerical  parameters;  as  a consequence,  their  main  disadvantage  is  the  lack  of  precision  in  the  results,  especially 
because  very  often  the  combination  of  "positive"  and  "negative"  influences  leads  to  "unknown"  relations.  The  motivation 
for  this  approach  is  based  on  the  fact  that  people  usually  reason  in  qualitative  terms. 

In  our  case,  we  used  the  Thurstone  analysis  as  a starting  point  to  identify  relative  positions  of  variables  of 
virtual  communities,  and  we  then  used  qualitative  reasoning  to  subjectively  identify  those  variables  that  are  of 
interest  and  influence  in  the  model  and  isolate  those  that  are  less  likely  to  have  an  impact  on  the  overall  performance 
of  the  model.  We  caution  the  reader  that  our  reasoning  is  based  on  our  teaching  and  research  experience  into  virtual 
learning  environments,  and  it  may  contain  epistemological,  contextual  and  personal  bias.  However,  the  initial 
precision  of  the  relationships  among  variables  is  less  important  to  developing  a model  than  is  the  identification  of 
key  variables  that  was  accomplished  by  using  the  grounded  theory  approach  mentioned  earlier.  Precision  is  built  by 
tuning  the  model  and  observing  how  variables  interact  over  time  and  across  contexts  in  the  BBN.  In  other  words,  a 
BBN  is  built  iteratively,  and  as  the  number  of  iterations  increase,  the  model  is  tuned  to  render  an  increasingly 
accurate  network  of  relationships  among  key  variables 

In  this  study,  we  used  qualitative  reasoning  to  infer  causal  relationships  among  the  variables  identified  in  the 
study,  resulting  in  relationships  among  variables  that  could  be  charted.  For  instance  one  can  qualitatively  and 
inductively  reason  that  in  virtual  learning  communities,  participation  and  learning  are  essentially  variables  whose 
interactions  are  mediated  by  another  technology  as  another  variable,  (i.e.,  it  is  hard  to  imagine  learning  online 
without  any  participation  and  equally  participation  is  often  mediated  by  technology),  and  therefore,  technology  is 
assigned  to  be  a parent  of  participation.  Similarly,  participation  can  influence  awareness  in  various  ways,  which  in 
turn  can  lead  to  the  development  of  trusting  relationships.  Since  awareness  can  contribute  to  trust  and  distrust,  trust 
is  set  to  be  a child  of  awareness. 

Furthermore,  one  can  reason  that  technology  influences  awareness  in  different  ways.  For  example,  imagine  a 
learning  environment  in  which  each  individual  has  a profile  (electronic  portfolio)  and  the  information  is  made 
available  to  others  in  the  community;  this  can  create  sense  of  awareness  about  who  is  who,  or  who  knows  what,  in 
that  community.  Similarly,  technology  may  influence  intensity  in  a weak  positive  manner.  For  example,  poor 
technology  might  have  negative  outcomes  on  engagement.  In  other  words,  people  might  not  be  willing  to  use 
technology  that  does  not  work  well  for  them,  or  they  find  awkward  to  use. 

Extending  this  type  of  qualitative  reasoning  resulted  in  the  BBN  shown  in  Figure  2.  In  the  model,  those  nodes 
that  contribute  to  causality  align  themselves  in  “parent”  to  “child”  relationships,  where  parent  nodes  are  causes  and 
child  nodes  are  effects.  For  example,  trust  is  the  child  of  mutuality;  awareness  and  intensity,  which  are  in  turn 
children  of  participation  and  technology  (see  Figure  2).  The  criterion  for  determining  causality  among  the  variables 
is  a reflection  of  our  qualitative  reasoning  process  (soft  data),  which  can  be  validated  using  empirical  evidence  (hard 
data). 
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Figure  2.  BBN  representation  of  relationships  among  virtual  learning  community  variables. 

The  third  step  in  building  the  model  involved  assigning  initial  probabilities  to  the  network.  In  general,  BBN 
initial  probabilities  can  be  obtained  from  domain  experts,  secondary  statistics  or  they  can  be  taken  from  observations 
and  subjective  intuition.  It  is  also  possible  that  initial  probabilities  can  be  learned  from  raw  data.  In  addition  to 
learning  prior  probabilities,  it  is  sometimes  necessary  to  examine  the  structure  of  the  network.  In  our  case,  the  initial 
probabilities  were  obtained  using  approached  discussed  in  Daniel,  Zapata-Revera  and  McCalla  (2003)  and  the 
structure  and  the  degree  and  strength  of  influence  among  the  variables  was  determined  by  examining  the  distances 
between  the  variables  of  virtual  learning  communities  along  the  Thurstone  Scale.  This  approach  enabled  us  to 
cluster  those  variables  that  were  closely  aligned  on  the  Thurstone  scale  and  use  weighted  threshold  values  (Daniel, 
McCalla,  & Schwier,  2005)  to  generate  the  conditional  probability  table.  The  relationships  and  the  degree  of 
influence  among  the  variables  were  further  described  qualitatively.  In  the  results  of  Thurstone  scaling,  those 
variables  that  cluster  around  the  mean  scale  point  was  observed  were  given  high  degree  of  influence. 

Generating  the  Conditional  Probability  Values 

The  initial  conditional  probabilities  were  generated  by  examining  qualitative  descriptions  of  the  influence 
between  two  or  more  variables  and  the  strength  of  their  relationships  in  the  model  (Daniel,  Zapata-Rivera,  McCalla, 
2003;  Daniel,  McCalla,  & Schwier,  2005).  Each  probability  describes  the  strength  of  relationship.  For  instance, 
various  degrees  of  influence  among  variables  are  represented  in  the  model  by  the  letters  S (strong),  M (medium), 
and  W (weak).  The  signs  + and  - represent  positive  and  negative  relationships.  The  elicitation  of  the  initial 
probability  approach  for  the  variables  was  based  on  the  approach  discussed  in  Daniel,  Zapata-Revera  and  McCalla 
(2003),  but  the  strengths  of  the  relationships  and  the  influence  of  each  variable  was  based  on  the  results  of  the 
Thurstone  scaling  and  the  relative  positioning  of  each  variables  along  the  scale.  For  instance,  technology  was  ranked 
to  be  last  and  so  it  carries  a threshold  probability  value  of  0.6  and  the  symbol  weak  (W+)  was  assigned  to  it.  The 
sign  means  that  there  is  some  kind  of  influence,  but  because  of  its  low  ranking  along  the  scale,  the  influence  is  a 
weak  one. 

The  probability  values  were  obtained  by  adding  weights  to  the  values  of  the  variables  depending  on  the  number 
of  parents  and  the  strength  of  the  relationship  between  particular  parents  and  children.  For  example,  if  there  are 
positive  relationships  between  two  variables,  the  weights  associated  with  each  degree  of  influence  are  determined  by 
establishing  a threshold  value  associated  with  each  degree  of  influence.  The  threshold  values  correspond  to  the 
highest  probability  value  that  a child  could  reach  under  a certain  degree  of  influence  from  its  parents,  i.e.  assuming 
that  Participation  and  Technology  have  positive  and  strong  relationships  with  Awareness,  evidence  of  good 
technology  and  high  participation  will  result  into  a conditional  probability  value  of  0.98  (i.e.,  Awareness=Exist). 

This  value  is  obtained  by  subtracting  a base  value  (1  / number  of  parents— 0.5  in  this  case  with  two  parents)  from  the 
threshold  value  associated  to  the  degree  of  influence  (i.e.,  threshold  value  for  strong  = 0.98)  and  dividing  the  result 
by  the  number  of  parents  (i.e.,  (0.98  - 0.5)  12  = 0.24).  Table  3 lists  threshold  values  and  weights  used  in  this 
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example.  The  value  a = 0.02  leaves  some  room  for  uncertainty  when  considering  evidence  coming  from  positive 
and  strong  relationships. 


Table  2.  Threshold  values  and  weights  with  two  parents 


Degree  of  influence  Thresholds  Weights 

Strong  l-cc=  1 - 0.02  = 0.98  (0.98-0.5)  /2  = 0.48  / 2 = 0.24 

Medium  0.8  (0.8-0.5)  / 2 =0.3  / 2 = 0. 15 

Weak  0.6  (0.6-0.5)  / 2 =0.1  / 2 = 0.05 


Daniel,  Zapata-Revera  and  McCalla  (2003) 

This  assumes  that  participation  and  technology  have  positive  strong  relationships  with  awareness  and  there  is 
evidence  of  positive  participation  and  technology  in  a particular  community.  Given  these  assumptions,  weights  will 
be  added  to  the  conditional  probability  table  of  awareness  every  time  participation  = high  or  technology  = good.  For 
example,  the  conditional  probability  value  associated  with  awareness  given  that  there  is  evidence  of  participation  = 
high  and  technology  = good  is  0.98.  This  value  is  obtained  by  adding  to  the  base  value  the  weights  associated  with 
participation  and  technology  (0.24  each).  Table  4 shows  a complete  conditional  probability  table  for  this  example. 

Table  3.  An  example  of  conditional  probability  table  for  two  parents  with  strong,  positive  relationships 


Participation 
Technology 
Awareness  Exists 
Awareness  Does  Not  Exist 


High 

Low 

Good 

Bad 

Good 

Bad 

0.98 

0.74 

0.74 

0.5 

0.02 

0.26 

0.26 

0.5 

The  calculation  of  the  various  states  of  the  relationships  among  the  three  variables  (awareness,  participation  and 
technology),  and  their  corresponding  values  used  in  Table  3.  Given  below: 

P (Awareness=  Exist  \ Participation = high  & Technology=  Good)  = 0.5  + 0.24  + 0.24  = 0.98 
P (Awareness=  DoesNotExist \ Participation=  high  & Technology=  Good)  = 1 - 0.98  = 0.02 
P (Awareness=  DoesNotExist  \ Participation=High  & Technology=  Bad)  = 1 - 0.74  = 0.26 
P (Awareness=  Exist \ Parti cipation=  Low  & Technology=  Good)  = 0.5  + 0.24  = 0.74 
P (Awareness=  DoesNotExist  \ Participation=  Low  & Technology=  Good)  = 1-0.74  =0.26 


Querying  the  Network 

Querying  a BBN  refers  to  the  process  of  updating  the  conditional  probability  table  and  making  inferences  based 
on  new  evidence.  One  way  of  updating  a BBN  is  to  develop  a detailed  number  of  scenarios  that  can  be  used  to  query 
the  model.  A scenario  refers  to  a written  synopsis  of  inferences  drawn  from  observed  phenomenon  or  empirical  data. 
Druzdzel  and  Flenrion  (1993)  described  a scenario  as  an  assignment  of  values  to  those  variables  in  Bayesian  network 
which  are  relevant  for  a certain  conclusion,  ordered  in  such  a way  that  they  form  a coherent  story — a causal  story 
which  is  compatible  with  the  evidence  of  the  story.  The  use  of  scenarios  in  Bayesian  network  is  drawn  from 
psychological  research  (Pennington  & Flastie,  1988).  This  research  shows  that  humans  tend  to  interpret  and  explain 
any  social  situation  by  weighing  up  the  most  credible  stories  that  include  hypotheses  to  test  and  understand  social 
phenomena.  In  Bayesian  terms  a hypothesis  is  the  assignment  of  a value  to  a discrete  variable  or  group  of  variables. 
Although  a scenario  can  describe  all  of  the  nodes  a model,  it  is  more  reasonable  to  include  only  the  nodes  relevant 
for  a certain  situation.  If  there  is  a certain  focal  hypothesis,  say  for  instance  H,  selected  by  the  user,  the  relevant 
nodes  are  those  that  affect  the  posterior  probability  of  H given  the  observed  evidence  e.  Otherwise,  the  relevant 
nodes  are  all  those  whose  probabilities  depend  on  e.  The  explanation  of  the  model  therefore,  consists  of  showing  the 
evidence  (i.e.,  the  scenarios  that  are  most  compatible  with  the  hypothesis  and  those  that  are  incompatible  with  the 
hypothesis. 
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Furthermore,  updating  a BBN  using  scenarios  is  an  attempt  to  understand  the  significance  of  various 
relationships  among  variables  in  a network.  Based  on  the  results  of  Thurstone  scaling  we  have  observed  a large 
cluster  of  variables  around  the  mean  scale  point.  These  were  then  chosen  to  construct  the  Bayesian  Belief  network. 
Although,  the  variables  obtained  in  the  earlier  analysis  can  be  treated  as  a group  because  of  their  central  position 
relative  to  the  other  points,  it  is  difficult  to  measure  their  individual  importance  relative  to  others  in  the  same  cluster 
or  in  other  clusters  in  the  VLC  model.  We  build  simple  scenarios  to  further  infer  their  relative  influence  and 
significance  to  learning  within  the  network.  The  approach  described  in  table  3 uses  both  qualitative  and  quantitative 
data  in  building  the  Bayesian  Network  to  model  imprecise  and  nebulous  domains  (Daniel,  Zapata-Revera  & 
McCalla,  2003).  In  addition,  the  probability  distribution  enables  us  to  query  the  model  and  observe  changes  as  they 
propagate  to  generate  new  posterior  probability  values  (P),  which  we  can  then  use  to  make  logical  inferences  about 
the  state  of  the  model  from  changes  in  its  variables.  For  example,  imagine  a community  where  there  is  reasonably 
high  level  of  participation  among  individuals  (e.g.,  p=0.98),  and  a high  presence  of  mutuality,  implying  learners  are 
constantly  engaged  in  reciprocal  relationships  through  exchanging  messages,  sharing  experiences,  stories, 
information  and  knowledge.  Querying  the  model  (presented  in  figure  2)  with  this  scenario  reveals  increased  learning 
with  a posterior  probability  value  of  P (1=0.763). 

Another  scenario  we  employed  to  tune  the  model  involved  a formal  virtual  learning  community  in  which  an 
effective  level  of  participation  guided  by  explicit  social  protocols  was  observed.  In  addition,  individuals  were 
constantly  engaged  in  open  discourse  (mutuality),  and  the  issues  were  addressed  in  both  depth  and  breadth 
(intensity).  Further,  assuming  that  there  is  a high  intensity  in  discourse  encouraged  individuals  to  reflect  deeply  on 
the  issues  being  discussed.  Results  of  querying  the  model  using  this  scenario  revealed  a higher  probability  of 
learning  p (1=0.779)  with  a significant  difference  of  0.016  compared  to  the  probability  of  learning  in  the  presence  of 
effective  participation  and  mutuality  alone.  This  result  is  intuitively  appealing,  given  interview  data  that  suggested 
that  a combination  of  these  factors  encouraged  depth  in  the  discussion  and  in  learning  (Schwier  & Daniel,  2006). 

In  practice  virtual  learning  communities  should  encourage  freedom  of  expression,  mutual  respect  and  they 
should  value  diversity.  Building  on  the  notion  of  individual  freedom  in  a virtual  learning  community,  we  were 
interested  in  observing  the  impact  of  autonomy  on  trust  and  learning,  given  effective  participation  and  good 
technology.  Autonomy  seems  to  be  very  influential;  the  network  revealed  higher  probability  of  trust  P (t=0.924)  and 
correspondingly  high  probability  of  learning  P (1=0.794)  when  autonomy  was  elevated. 

Given  the  central  importance  of  trust  as  a prerequisite  condition  of  learning,  we  were  interested  in 
understanding  the  impact  of  all  the  variables  on  trust  and  learning.  In  this  scenario  all  the  variables  in  the  first  layer 
(technology  and  participation)  and  second  layer  (mutuality,  intensity,  social  protocols,  reflection,  autonomy  and 
awareness)  in  the  model  were  set  to  their  highest  probability  values.  This  scenario  increased  the  values  of  posterior 
probabilities  of  trust  (P:  (t=0.944)  and  learning  (P:  1=0.810).  This  result  suggests  that  the  variables  in  the  network 
can  collectively  have  considerable  and  yet  varying  effects  on  trust  and  learning,  depending  on  differing  scenarios. 

Although  the  results  of  the  Thurstone  analysis  ranks  trust  to  be  the  most  important  variable  in  a virtual  learning 
community,  our  analysis  suggests  that  when  trust  is  associated  directly  with  learning,  but  without  the  positive 
influence  of  its  parent  variables  (mutuality,  intensity,  social  protocols,  reflection,  autonomy  and  awareness),  the 
probability  of  learning  remains  low  P (1=0.629).  This  result  holds  even  in  the  presence  of  good  technology  and 
effective  participation.  Previous  research  has  emphasized  the  value  of  trust  in  enhancing  the  sense  of  a community. 
Prusak  and  Cohen  (2001)  suggested  that  trust  enables  people  to  work  together,  collaborate,  and  smoothly  exchange 
information  and  share  knowledge  without  time  wasted  on  negotiation  and  conflict.  In  virtual  learning  communities 
however,  we  argue  that  without  mutuality,  intensity,  social  protocols,  reflection,  and  awareness;  the  impact  of  trust 
on  learning  may  be  minimal. 

Based  on  different  experiences,  experts’  knowledge,  intuition  and  hunches,  a large  number  of  scenarios  can  be 
developed  to  query  this  model.  Querying  the  model  using  logical  scenarios,  whether  based  on  empirical  data  or 
experts’  experiences,  offers  a disciplined  method  of  examining  the  cumulative  effect  of  making  changes  anywhere 
in  the  network  and  also  for  speculating  about  how  any  particular  change  can  alter  the  values  of  related  variables.  The 
BBN  is  still,  at  its  core,  a tool  for  speculation,  but  over  time  and  as  data  are  added  to  inform  the  variables  and  their 
interrelationships,  the  network  can  be  "tuned"  to  provide  robust  and  precise  ways  to  make  decisions  about  how  to 
support  learning  in  virtual  learning  communities 
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Conclusions 


The  most  important  point  in  this  paper  is  not  the  specific  methods  we  chose  to  perform  the  analyses.  A host  of 
other  tools  are  available  to  researchers,  and  they  can  be  used  to  generate  related  but  different  interpretive  angles  on 
data  (e.g.,  social  network  analysis  tools).  Of  greater  significance  is  that  the  methods  flow  from  definition  to  analysis 
to  prediction,  so  they  have  some  intuitive  and  practical  appeal.  We  recognize  that  we  are  at  the  beginning  of 
learning  about  how  to  understand  online  learning  communities,  and  so  we  make  no  claims  that  these  methods 
represent  a definitive  set  of  tools  for  that  job,  but  we  think  that  considering  the  full  cycle  from  definition  to  modeling 
is  important.  Much  of  the  research  to  date  that  examines  online  learning  communities  looks  closely  at  only  a few 
variables,  and  much  of  the  literature  is  highly  speculative.  We  think  that  there  is  a need  to  systematically  isolate 
features  of  communities,  try  to  determine  their  relative  importance,  and  then  build  models  that  can  be  used  to  test 
inferences  in  new  environments  and  inform  the  science  of  design  in  online  learning. 
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Programs  that  prepare  school  library  media  specialists  face  many  challenges  in  the  current  climate.  Despite 
the  large  body  of  evidence  of  the  considerable  benefits  of  strong  school  library  media  programs  situated  at  the  center 
of  each  school’s  learning  community  (e.g.  Lance,  Welborn  and  Hamilton-Pennell,  1993),  we  appear  to  be  constantly 
fighting  to  maintain  our  position.  We  see  funds  being  cut,  media  center  budgets  being  taken  out  of  the  media 
centers,  a lowering  of  media  staffing  levels,  or  the  elimination  of  media  specialist  positions.  We  see  pressure  put  on 
media  specialists  to  act  as  computer  repair  workers,  as  software  troubleshooters,  as  technology  police,  and  as  baby 
sitters.  Under  these  circumstances  it  is  a constant  battle  to  maintain  the  integrity  of  our  programs  and  at  the  same 
time  keep  them  relevant  to  the  world  in  which  our  graduates  will  work. 

Where  do  we  invest  our  limited  preparation  time?  In  traditional  library  preparation  courses  such  as 
cataloging,  reference,  and  selection?  In  preparation  for  teaching  information  literacy  and  classroom  management?  In 
technical  courses  for  media  center  management  or  for  technical  support  of  networks  and  software  and  hardware 
troubleshooting?  In  advocacy  and  education  of  principals  and  teachers,  school  boards,  legislators,  and  parents?  And 
how  do  we  strike  a balance  among  these  disparate  areas  that  both  upholds  our  professional  standards  and  at  the  same 
time  gives  adequate  preparation  to  our  students  to  enable  them  to  walk  into  a media  center  and  perform  this  exacting 
job  to  a high  level.  This  paper  will  discuss  some  of  the  ways  in  which  one  master’s  level  initial  preparation  program 
in  school  library  media  in  a large  urban  setting  endeavored  to  take  stock  of  program  strengths  and  weaknesses. 

The  program  in  question,  based  in  the  college  of  education  of  a large  urban  research  university,  has  a 
substantial  35  year  history  of  producing  well-qualified  media  specialists,  prepared  to  a high  standard  in  their  field. 
The  State  of  Georgia  has  until  recently  used  the  Praxis  II  Subject  Test  (Media  Specialist)  as  its  certification  exam, 
and  has  the  second  highest  required  passing  score  in  the  nation  (620  points,  with  only  Maryland  and  Oregon 
requiring  a higher  qualifying  score  of  630  points.)  During  the  last  five  years  for  which  data  are  available,  the 
program  has  a 100%  pass  rate,  with  the  majority  of  the  students  achieving  scores  in  the  660  - 750  range.  (Georgia 
State  University,  2006b). 

Five  years  ago  the  program  was  at  a low  ebb  from  a staffing  point  of  view.  Several  retirements  had  reduced 
the  full  time  faculty  to  two  in  number.  In  common  with  many  other  programs,  including  those  in  library  science, 
recruitment  of  new  faculty  proved  to  be  a major  hurdle,  and  several  faculty  searches  were  concluded  unsuccessfully. 
The  program  continued  to  be  staffed  by  two  full  time  and  six  adjunct  faculty  until  a catastrophic  illness  reduced  the 
full  time  faculty  to  one  person.  At  this  point  an  aggressive  recruiting  campaign  was  mounted  and  three  highly 
qualified  faculty  members  were  appointed,  with  a fourth  joining  the  faculty  two  years  later. 

During  the  first  year  of  the  new  faculty’s  appointment,  the  department  in  which  the  program  was  housed 
was  slated  to  conduct  its  seven  year  internal  review.  In  preparation  for  this  the  faculty  began  a review  process  where 
all  program  offerings  were  studied  both  individually  and  as  a whole  and  compared  with  national  and  state  standards. 
At  this  time  the  State  of  Georgia  Professional  Standards  Commission  used  the  American  Library 
Association/ American  Association  of  School  Librarians  standards  for  the  prep  of  school  librarians  (ALA/AASL 
2004)  as  the  basis  for  certification.  The  standards  of  the  Interstate  New  Teacher  Assessment  and  Support 
Consortium  (INTASC)  were  also  considered,  where  appropriate.  Several  courses  were  changed  in  light  of  this 
review,  some  substantially,  others  less  so,  with  some  outdated  content  being  removed  to  make  way  for  increased 
emphasis  on  technology  integration  and  the  teaching  of  information  literacy.  More  emphasis  was  added,  for 
example,  to  the  study  of  electronic  resources  and  networked  information  systems  in  the  basic  reference  services 
class.  Several  classes  such  as  that  in  instructional  collaboration  began  not  only  to  require  the  use  of  electronic 
presentation  and  communication,  but  to  teach  presentation  techniques  and  basic  public  speaking.  Although  a field 
based  internship  is  required  of  students,  many  smaller  field  experiences  were  added  to  all  courses  to  strengthen  the 
practical  elements  of  the  program,  as  part  of  this  standards-alignment  process. 
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Although  these  program  changes  were  the  result  of  thoughtful  consideration  by  faculty  from  teaching  the 
courses,  of  recommendations  from  the  external  reviewers  at  the  end  of  the  departmental  review,  and  of  planning  in 
light  of  professional  standards,  they  had  not  been  examined  in  light  of  the  students  or  the  workplace.  Accordingly 
three  major  assessments  were  planned  to  gauge  the  effectiveness  of  these  changes  and  to  seek  additional  areas  for 
program  improvement.  The  first  of  these  involved  students  currently  in  the  program,  the  second  students  who 
graduated  within  the  last  12  months,  and  the  third  media  coordinators  in  the  school  systems  served  by  our  program. 

Over  the  last  few  years  teacher  education  programs  have  recognized  the  importance  of  reflective  writing  on 
professional  development.  This  program  has  now  instituted  a Professional  Development  Reflection  as  part  of  its 
continuing  assessment.  Each  semester  students  are  required  to  complete  a reflection  on  their  learning  experiences 
throughout  the  semester.  The  reflection  covers  all  coursework  attempted  that  semester,  as  well  as  all  field 
experiences.  Students  are  encouraged  to  look  across  their  coursework  to  identify  common  themes,  to  see  each  course 
as  one  part  of  a complex  whole,  to  identify  areas  where  they  feel  well  prepared,  as  well  as  areas  of  weakness.  This 
has  led  to  students  developing  a stronger  framework  into  which  the  various  course  elements  can  be  fitted  as  each 
course  is  taken. 

An  analysis  of  the  available  reflections  shows  some  interesting  trends.  As  students  have  worked  through 
successive  semesters,  the  depth  of  their  professional  reflections  deepens,  and  they  are  more  able  to  put  their  learning 
into  a suitable  context.  The  increase  in  the  number  of  field  experiences  has  enabled  students  to  see  more  media 
centers  in  operation  and  more  media  specialists  at  work,  and  this  has  enabled  them  to  develop  a more  realistic 
perspective  of  the  current  role  of  the  media  specialist,  and  the  common  difference  between  how  the  role  of  Teacher- 
Librarian  is  described  for  example  in  Information  Power  (1998)  and  that  commonly  found  in  many  schools  where 
the  media  specialist  is  viewed  simply  as  non-professional  support  staff.  It  has  also  given  them  more  relevant  criteria 
by  which  to  judge  their  own  learning  and  preparedness  to  enter  the  profession,  and  reflections  suggest  that  the 
students  are  increasingly  aware  of  the  complexities  of  their  chosen  profession  and  their  level  of  skill  attainment. 

The  use  of  these  professional  development  reflections  for  program  evaluation  is  only  just  beginning  to  be 
realized.  Initially  the  faculty  thought  the  reflections  would  identify  areas  that  were  lacking  in  the  curriculum,  but  this 
has  proved  to  be  case  only  on  a very  limited  level.  Despite  the  number  of  field  experiences,  students  are  not  always 
in  a position  to  know  what  they  do  not  know,  especially  at  the  pre-service  stage.  Some  of  the  procedures  or  practices 
viewed  in  the  media  centers  have  turned  out  to  be  special  requirements  or  practices  of  individual  school  systems, 
rather  than  more  generalizable  best  media  center  practice,  and  while  valuable  for  students  to  observe,  are  not 
necessarily  suitable  for  inclusion  in  the  professional  preparation  program.  (It  may  also  be  the  case  that  the  many 
curriculum  revisions  that  were  made  as  part  of  the  initial  internal  review  strengthened  the  curriculum  to  the  point 
where  the  major  gaps  had  already  been  identified  and  filled.) 

However  the  value  of  these  reflections  lies,  for  evaluation  purposes,  in  seeing  how  the  students  are  reacting 
to  changes  made  to  curriculum  and  how  they  are  developing  professionally  and  assimilating  their  new  knowledge. 

To  a large  extent  this  use  depends  on  a long  term  look  at  the  professional  development  reflections,  and  the  program 
is  only  now  completing  its  first  year  of  student  reflections.  One  area  for  improvement  in  the  reflections  that  has  been 
identified  is  the  idea  of  having  the  students  use  professional  standards  as  the  basis  for  their  reflections.  Rather  than 
simply  reflecting  on  what  they  already  know,  the  standards  will  give  students  a stronger  framework  to  the 
reflections,  and  better  guidance  as  to  the  final  goals  of  the  program.  Another  area  of  improvement  in  this  process  is 
to  encourage  new  graduates  to  continue  their  professional  reflections  and  to  share  that  information  with  the  program. 
The  reflections  of  new  media  specialists  as  they  adapt  to  life  on  the  job  will  yield  a better  picture  of  areas  where 
students  feel  well-prepared  and  those  where  additional  preparation  is  necessary. 

Of  particular  interest  is  the  difference  in  reflections  among  the  various  types  of  students  in  the  program. 
The  Georgia  Professional  Standards  Commission  (2006)  does  not  require  that  media  specialists  hold  teaching 
certification,  so  there  are  three  major  categories  of  students  within  the  certification  program:  certified  teachers 
seeking  to  add  a media  specialist  qualification,  public  librarians  seeking  to  become  certified  for  school  media 
centers,  and  those  from  outside  either  education  or  librarianship  seeking  initial  certification  as  school  media 
specialists.  The  majority  of  students  fall  into  the  first  category,  so  it  is  only  gradually  that  we  will  be  able  to  see  a 
tmer  picture  of  the  needs  of  the  other  two  categories  of  students. 

The  original  plan  for  this  program  evaluation  called  for  an  online  questionnaire  of  graduates  of  the  program 
within  the  previous  12  months  based  on  the  ALA/AASL  standards  and  the  Praxis  II  Subject  Test  (Media  Specialist), 
with  a corresponding  questionnaire  to  county  media  coordinators.  At  the  time  of  planning  these  were  the  standards 
and  the  certification  test  accepted  by  the  State  of  Georgia  and  required  by  the  university  for  degree  and  certification 
program  purposes  (Georgia  State  University  2006a).  However,  with  effect  from  January  2006,  Georgia  adopted  its 
own  set  of  professional  standards  for  the  preparation  of  school  media  specialists  (Georgia  Professional  Standards 
Commission,  2006),  at  the  same  time  announcing  a new  series  of  certification  tests,  the  Georgia  Assessment  for  the 
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Certification  of  Educators  (GACE)  and  the  withdrawal  of  the  Praxis  II  series  in  most  fields.  It  seemed  pointless  to 
proceed  with  questionnaires  based  on  standards  that  were  no  longer  governing  preparation  programs  within  the  state, 
or  on  assessment  tests  that  may  have  a different  focus  from  those  originally  planned  as  the  basis  for  the  evaluation 
instalments.  These  parts  of  the  evaluation  are  now  in  preparation,  although  some  delay  is  still  expected  as  the  GACE 
tests  will  only  be  administered  for  the  first  time  in  late  fall  2006.  Both  faculty  and  students  from  the  program  were 
involved  in  the  development  and  field  testing  of  this  new  test,  but  we  will  not  have  a clear  picture  of  the  actual 
emphasis  of  the  test  and  how  it  assesses  the  various  sub  areas.  In  contrast  to  the  Praxis  II  Subject  Test  (Media 
Specialist  which  has  five  concentration  areas  (program  administration;  collection  development;  information  access 
and  delivery;  learning  and  teaching;  and  professional  development,  leadership  and  advocacy),  the  GACE  test  has 
four  (information  access  and  delivery  in  the  library  media  program;  information  literacy  skills;  collaboration  and 
leadership  in  the  library  media  context;  and  library  media  program  administration.)  It  is  not  yet  clear  whether 
changes  in  curriculum  will  be  required  to  meet  the  expectations  of  this  new  test,  but  evaluation  will  be  conducted  as 
soon  as  is  practicable. 

As  the  world  of  the  media  specialist  changes  and  increasing  demands  are  made  on  the  profession,  it  is 
imperative  that  initial  preparation  programs  ensure  they  are  producing  graduates  who  are  not  only  skilled  in  the 
traditional  librarian  aspects  of  the  job,  not  only  prepared  to  teach  information  literacy  as  a teaching  professional,  but 
who  are  also  competent,  flexible  professionals  who  are  capable  of  creating  and  running  school  media  programs  that 
function  as  the  instructional  center  of  the  school.  Only  by  continuous  assessment  of  program  content  and 
presentation  can  faculty  monitor  program  effectiveness  and  serve  both  students  and  school  districts  to  maximum 
effect. 
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Abstract 

This  paper  features  one  of  the  most  dynamic  instructional  models  used  in  higher  education  in  recent  years. 
Synthesizing  aspects  drawn  from  the  STAR  Legacy  cycle  and  the  VaNTH  model,  an  instructional  design  model  is 
conceptualized,  graphically  portrayed,  and  illustrated  by  example.  The  latest  tools  available  to  online  learners  — 
conference  management  systems  — are  integrated  and  demonstrated  using  two  sample  activities  from  one 
Educational  Technology  foundations  course  and  one  Instructional  Systems  Development  course.  Lessons  learned  of 
this  joint  effort  are  also  outlined. 


Introduction 

An  ongoing  problem  for  educators  is  to  find  meaningful  ways  to  keep  learners  actively  involved  in  an 
instructional  task.  Keeping  students  involved  has  been  labeled  “active  learning.”  While  most  educators  see  the 
inherent  value  in  having  the  student  involved  in  active  learning,  most  educators  would  also  readily  agree  that 
designing  and  implementing  instruction  that  meaningfully  engages  the  learner  is  no  trivial  undertaking  for  learners 
of  any  age.  In  the  present  paper,  we  would  like  to  briefly  introduce  the  “How  People  Learn”  model  and  exemplify 
the  learning  model  using  two  sample  activities  from  two  graduate  courses  from  a completely  online  Education 
Technology  program.  We  conclude  this  paper  by  reinforcing  major  aspects  of  the  implementation  of  the  model  and 
addressing  some  of  the  challenges  we  encounter  in  a real  learning  context. 

STAR  Legacy  Cycle 

There  has  been  significant  progress  made  in  the  last  couple  of  decades  in  determining  what  is  involved  in 
the  learning  process,  and  how  educators  can  go  about  creating  and  implementing  engaging  instruction  for  learners. 

A seminal  book  in  this  area  entitled,  How  People  Learn,  is  authored  by  Bransford,  Brown,  and  Cocking  (2000).  The 
book  has  become  a highly  regarded  tool  for  helping  educators  of  all  stripes  design  instruction  based  upon  proven 
principles  of  how  to  meaningfully  engage  learners.  How  People  Learn  (HPL)  has  been  used  to  create  a cycle,  or 
framework,  coined  the  STAR  Legacy  (Software  Technology  for  Action  and  Reflection)  cycle  (Schwartz,  Brophy, 
Xiaodong,  & Bransford,  1999).  This  framework  is  a useful  guide  for  designing  instruction  that  is  intended  to 
actively  engage  the  learner.  The  cycle  comprises  a sequence  of  six  consecutive  components:  Challenge,  Generate 
Ideas,  Gather  Multiple  Perspectives,  Research  & Revise,  Test  Your  Mettle,  and  Go  Public. 

The  cycle  can  be  further  modified  to  include  specifying  course  objectives  and  designing  meaningful 
challenges  based  upon  those  objectives.  The  guideline  then  becomes  the  following  steps,  taken  from  a hybrid  model, 
a.k.a.,  VaNTH,  developed  by  a consortium  of  several  of  the  engineering  research  centers  from  some  of  the  leading 
universities  in  the  United  States,  (see  http://www.vanth.org/). 

1.  Identify  course  objectives  - these  are  statements  of  what  a student  will  know  and  do  after  instruction. 
Students  see  how  knowledge  is  applied. 

2.  Design  challenges  for  instruction  - these  are  statements  that  pose  a complex  goal  to  the  students.  You,  as 
the  instructor/facilitator,  need  to  develop  interesting  challenges  that  engage  students  in  a process  of  inquiry 
that  requires  them  to  apply  the  desired  concepts  beyond  simple  manipulation  of  mathematics,  for  example. 

3.  Generate  ideas  - this  step  provides  students  with  the  opportunity  to  explore  what  they  currently  know  about 
the  challenge.  This  includes  their  naive  concepts  or  models  of  the  domain  and  will  provide  a baseline  or 
pre-assessment  of  what  they  know  about  the  challenge. 

4.  Multiple  perspectives  - these  are  statements  by  experts  describing  what  they  see  in  the  challenge.  Their 
comments  provide  insights  into  various  dimensions  of  the  challenge,  but  do  not  provide  a direct  solution  to 
the  challenge.  Students  can  compare  their  initial  thoughts  with  the  experts. 

5.  Research  and  revise  - this  component  provides  a series  of  learning  activities,  such  as  simulations,  lectures, 
homework,  labs,  and  readings,  designed  to  help  students  focus  on  the  important  dimensions  of  the 
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challenge.  These  activities  are  designed  to  help  the  students  make  a link  to  the  original  challenge 
aforementioned. 

6.  Test  your  mettle  - this  assessment  method  provides  students  the  opportunity  to  apply  what  they  know  and 
evaluate  what  they  need  to  study  more.  It  also  allows  the  students  to  reflect  on  how  well  they’ve  learned  the 
content  and  to  evaluate  if  they  are  ready  to  Go  Public,  which  is  the  next  step,  with  what  they  know. 

7.  Go  public  - this  is  the  final  assessment  of  what  students  know  at  the  end  of  the  module.  This  assessment 
could  be  a presentation  of  the  content. 

While  this  model  has  been  shown  to  be  one  of  the  effective  models  for  developing  instruction,  it  has  not  lent  itself  to 
the  design  and  development  of  online  instruction.  There  are  possibly  several  significant  barriers  that  prohibit  this 
model  from  being  used  in  the  development  of  online  instruction.  However,  the  most  apparent  obstacle  is  easy  to 
discern.  The  VaNTH  (Star  Legacy)  model  is  predicated  upon  a collaborative  effort  of  learners  who  engage  in  a 
significant  amount  of  interactivity.  One  can  easily  visualize  this  type  of  model  being  used  not  only  for  instructional 
purposes  in  more  progressive  classrooms  around  the  country,  but  also  in  any  workplace  that  requires  teams  of 
employees  to  conceptualize  and  solve  complex  problems.  But  the  challenge  in  implementing  this  model  in  designing 
and  implementing  online  instruction  seems,  at  first  glance,  overwhelming,  when  one  considers  the  simple  fact  that 
according  to  an  article.  The  Halo  Effect,  published  in  the  December  issue  of  The  Economist  magazine  in  2005 
(Katzenberg,  2005),  currently  90%  of  all  online  instruction  is  asynchronous  and  therefore  not  conducive  to  the 
ongoing  dialoguing  the  VaNTH  model  (or  Star  Legacy  Cycle)  requires. 

Practice 

Regardless,  we  managed  to  adopt  the  Star  Legacy  cycle  and  incorporate  it  into  the  activity  design  in  the 
first  two  required  courses  in  our  Educational  Technology  graduate  program.  Even  though  both  classes  emphasize 
more  (approximately  70%  of  the  final  grade)  on  the  cultivation  of  individual  competencies  for  being  a successful 
Educational  Technology  graduate,  we  decided  to  integrate  a cooperative  project  into  the  course  requirements  for  the 
following  reasons.  Our  current  curriculum  design  stresses  individual  achievement.  Workforce  situations  for  our 
graduates  require  our  students  routinely  work  in  authentic,  collaborative  situations.  The  other  reason  we  perceive  the 
need  for  students  to  engage  in  collaboration  to  solve  a common  goal  is  to  assist  our  students  in  fulfilling  one  of  the 
criteria,  simply  called  “Collaboration,”  required  in  the  development  of  an  electronic  portfolio  project  — a graduation 
requirement. 

The  Star  Legacy  model  is  introduced  in  the  beginning  of  the  class  in  a synchronous  class  meeting.  Prior  to 
the  cooperative  project,  students  in  both  classes  are  able  to  get  accustomed  to  the  learning  model  in  biweekly 
Horizon  Wimba  Group  Exercises.  Two  weeks  before  the  due  date  of  the  cooperative  project,  an  email  is  sent  to  the 
class  to  reinforce  the  concept  of  the  Star  Legacy  model. 

Both  classes  are  facilitated  in  Blackboard,  a course  management  system:  one  is  the  foundations  course,  and 
the  other  is  the  instructional  systems  development  (ISD)  course. 

The  foundations  course  is  intended  to  provide  a broad  overview  of  the  ideas  that  form  the  basis  of  the  field 
of  educational  technology.  While  this  course  is  not  entirely  a review  of  the  latest  technological  advances,  this 
introductory-level  course  does  present  an  overview  of  the  media  applications,  instructional  approaches  and  design 
processes  used  in  creating  performance  technology  solutions.  The  concepts  covered  are  generally  applicable  to  both 
public  and  higher  education,  and  various  forms  of  corporate  and  non-profit  training  endeavors.  The  textbook 
adopted  is  authored  by  G.  L.  Anglin  (1995),  entitled,  Instructional  Technology’:  Past,  Present,  and  Future.  Figure  1 
below  lays  out  the  framework  for  applying  the  Star  Legacy  model  in  the  foundations  class. 


Overview 

One  primary  purpose  of  Project  3 is  to  assist  you  in  conceptualizing  and  demonstrating  how  learning 
occurs  in  a non-co-located  (virtual  or  distributed)  group  setting.  Your  group  is  expected  to  proceed  with 
functionality  and  collaboration  in  accordance  with  the  Star  Legacy  model  (see 
http://iris.peabody.vanderbilt.edu/slm.html). 
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Objectives 

By  completing  this  collaborative  project,  you  will  be  able  to: 

1.  Practice  working  in  a collaborative  situation  in  which  direct  group  contact  (F2F)  is  not 
appropriate  for  any  of  a number  of  reasons.  Your  group  is  encouraged  to  use  a variety  of  tools 
to  develop  and  present  your  group  project. 

2.  Present  to  the  class  either  an  example  of  a group  plan  for  an  application  of  the  principles  of 
educational  or  instructional  technology  to  a specific  problem,  or  to  present  your  group's 
findings  on  some  topic  we  have  or  have  not  covered  in  the  text. 

Further  details  of  these  two  objectives  are  translated  into  three  presentation  options  below. 

Presenting  Your  Findings 

We  use  Florizon  Wimba  Live  Classroom  and  its  capabilities  for  linking  our  presentation  format.  Your 
group  uses  this  conference  management  system  to  meet  with  the  rest  of  the  class  and  to  present  major 
aspects  of  your  group  effort  as  you  guide  us  through  it  in  20-30  minutes.  We  do  not  want  an  information 
dump  during  the  presentation.  That  is,  present  major  aspects  and  save  all  the  rest  of  the  details  for  your 
online  document. 

Posting  Your  Document  File 

After  the  URL  of  your  Project  3 is  retrieved,  a group  representative  is  expected  to  post  the  URL  to  a 
designated  forum,  titled  Project  3 URLs,  24  hours  before  the  scheduled  presentation. 

Working  With  Your  Group 

Your  group  meets  primarily  in  a designated  Florizon  Wimba  Group  Deliberation  Room.  Each  group 
meeting  must  be  archived  for  the  instructor's  future  references.  It  is  highly  recommended  that  you  follow 
the  Star  Legacy  model  and  meet  through  audio  conferencing  and  other  features,  like  application  sharing, 
in  Florizon  Wimba.  You  are  advised  to  start  this  planning  as  early  as  possible  and  do  your  part  first. 

Your  team  may  want  to  determine  who  is  responsible  for  what  information.  To  do  so,  you  need  to  take 
all  it  can  to  get  to  know  your  teammates.  After  knowing  each  other's  knowledge,  skills,  and  attitudes, 
you  as  a team  may  start  the  division  of  labor.  You  can  select  someone  to  act  as  moderator  for  the  group 
during  the  online  discussions  and  presentations.  One  way  of  presenting  the  group's  findings  is  to  assign 
individuals  to  be  responsible  for  determining  who  the  intended  audience  will  be,  what  resources  will  be 
required,  and  what  assessment  activities  will  be  necessary  to  conduct.  Everyone  in  the  group  can 
contribute  to  the  research  (conceptual  background)  as  well  as  diagnosing,  designing,  and  developing  the 
assessment  activities.  Each  individual  will  be  responsible  for  presenting  their  portion  of  the  work  during 
the  presentation.  Do  your  part  first  and  do  it  well. 

After  the  oral  presentation,  each  group  or  individuals  is  expected  to  define  and  / or  defend  group  work 
by  posting  revised  work  under  the  initial  posting  before  the  last  class  day  for  a better  grade. 

Presentation  Options 

For  your  collaborative  project  you  may  select  one  of  three  options: 

Option  1 : Determine  a problem  members  of  your  group  may  share  and  demonstrate  knowledge  as  to  how 
some  sort  of  application  of  educational  or  instructional  technology  could  help  alleviate  that  problem. 

You  may  want  to  provide  a history  of  the  problem,  and  describe  how  the  use  of  technology  has  been 
used  to  help  remedy  the  problem  in  the  past,  present,  and  possibly  in  the  future.  Please  include  a 
conceptual  framework  (research)  of  your  problem.  The  problem  need  not  be  thought  of  as  having  a 
negative  connotation.  Your  group  may  see  merit  in  how  distance  education  can  be  used  to  boost  the 
number  of  students  enrolling  in  AP  Spanish  literature  classes,  for  example,  or  how  cafeteria  staff  can  use 
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various  media  to  increase  nutritional  awareness  of  students,  or  look  at  ways  to  provide  training  to 
librarians  in  searching  remote  databases.  Your  group  need  only  develop  a PLAN  for  developing  an 
instructional  solution  to  that  specific  problem.  This  plan  is  to  be  produced  in  a webpage  format. 

The  plan  should: 

• State  an  educational  or  instructional  problem  and  how  and  why  that  problem  is  defined  as  such 

• Describe  characteristics  of  who  the  intended  audience  will  be 

• Provide  a list  of  necessary  resources  (including  skills,  personnel,  hardware,  cost  analysis,  etc.) 

• Describe  activities,  and  appropriate  means  of  assessment, 

• Provide  a conceptual  background  to  the  educational  or  instructional  problem  (do  some  research) 

• List  bibliographic  citations  to  relevant  print  and  non-print  materials 

Option  2\  Your  group  can  also  select  a topic  we  have  covered  (or  will  soon  cover)  from  the  text.  Your 
group  can  present  more  recent  information  pertaining  to  that  topic,  and/or  how  developments  in 
education  and  society  at  large  have  vindicated,  or  nullified,  predictions  made  in  the  text  from  ten  years 
ago.  Your  presentation  is  expected  to  be  a webpage  format.  The  presentation  Web  site  should: 

• Present  the  content  for  your  presentation 

• Outline  relevant  research  in  the  field 

• Provide  links  to  supporting  Web  sites  (i.e.,  research  studies,  examples  of  applications,  etc.) 

• List  bibliographic  citations  to  relevant  print  and  Web-based  materials 

Option  3:  Your  group  comes  up  with  its  own  topic  that  we  have  not  considered  and  make  a proposal  for 
your  topic.  For  example,  you  might  look  at  legal  and  political  considerations  of  implementing  DE 
professional  certifications,  or  what  a typical  school  looks  like  in  the  year  2020  according  to  futurists. 

This  option  is  also  presented  in  a webpage  format  and  it  should: 

• Present  the  content  for  your  presentation 

• Provide  a conceptual  background  to  the  educational  or  instructional  problem  (facts,  figures, 
real-world  examples,  research  studies— do  some  research) 

• List  bibliographic  citations  to  relevant  print  and  Web-based  materials 

Click  the  link  attached  below  for  a sample  of  Project  #3. 


Figure  1.  The  Cooperative  Project  task  in  the  foundations  class. 

In  the  follow-up  course,  the  design  course  (or  ISD  course),  the  goal  of  this  course  is  for  the  learners  to  be 
able  to  design  and  develop  the  best  instruction  possible.  To  attain  that  goal,  the  class  systematically  proceeds 
through  a series  of  modules,  each  with  specific  objectives.  By  completing  these  modules  successfully,  the  students 
are  expected  to  demonstrate  the  following  three  skills  and  conceptual  knowledge: 

• Identify  the  theoretical,  experiential,  and  critical  perspectives  in  instructional  design  as  it  is  applied  to  a 
variety  of  educational  settings, 

• Design,  develop,  implement  and  evaluate  instruction  for  a specific  group  of  learners,  in  a specific 
environment, 

• Work  collaboratively  with  other  instructional  designers  in  a distributed  environment,  working  together  to 
achieve  specific  instructional  objectives  for  a distributed  audience. 

A textbook,  titled  The  systematic  design  of  instruction,  by  Dick  and  Carey  in  2004  is  used  as  the  class  text.  Figure  2 
below  lays  out  the  framework  for  applying  the  Star  Legacy  model  in  the  ISD  class. 


Overview 

This  Cooperative  Project  is  your  opportunity  to  demonstrate  competences  (skills  and  knowledge)  and  to 
develop  dispositions  you  have  acquired  in  the  course.  As  you  can  tell,  there  is  a certain  flexibility  for  you 
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and  your  group  to  present  any  current  issues  pertaining  to  ISD.  Please  be  advised  that  you  are  not 
requested  to  develop  the  actual  course/lesson/session,  but  please  tell  us  something  new  and  exciting.  A 
suggested  list  of  topics  is  provided  at  the  bottom  of  this  page. 

One  primary  purpose  of  Cooperative  Project  is  to  assist  you  in  conceptualizing  and  demonstrating  how 
learning  occurs  in  a non-co-located  (virtual  or  distributed)  group  setting.  Your  group  is  expected  to 
proceed  with  functionality  and  collaboration  in  accordance  with  the  Star  Legacy  model  (see 
http://iris.peabody.vanderbilt.edu/slm.html). 

Objectives 

By  completing  this  group  project,  you  will  be  able  to: 

1.  Practice  working  in  a collaborative  situation  in  which  direct  group  contact  (F2F)  is  not 
appropriate  for  any  of  a number  of  reasons.  Your  group  is  encouraged  to  use  a variety  of  tools 
to  develop  and  present  your  group  project. 

2.  Present  to  the  class  either  an  example  of  a group  plan  for  an  application  of  the  principles  of 
educational  or  instructional  technology  to  a specific  problem  or  your  group's  findings  on  some 
topic  we  have  or  have  not  covered  in  the  text  after  reviewing  empirical  sources. 

Presenting  Your  Findings 

We  will  use  Florizon  Wimba  Live  Classroom  and  its  capabilities  for  linking  our  presentation  format. 
Your  group  will  use  this  conference  management  system  to  meet  with  the  rest  of  the  class  and  to  present 
major  aspects  of  your  group  effort  as  you  guide  us  through  it  in  20-30  minutes.  We  do  not  want  an 
information  dump  during  the  presentation.  Those  major  aspects  are  expected  to  presented  using  a 
multimedia  presentation  file  (e.g.,  *.ppt,  *.html,  *.swf,  or  others).  That  is,  present  major  aspects  and  save 
the  rest  of  the  details  for  your  online  document. 

Posting  Your  Online  Document 

Your  entire  group  effort  (i.e.,  the  online  document)  is  to  be  submitted  in  a Web  format  (e.g.,  *.doc  or 
*.pdf)  and  stored  in  the  Komodo  server.  After  the  URL  of  your  Cooperative  Project  is  retrieved,  a group 
representative  is  expected  to  post  the  URL  to  a designated  forum,  titled  Cooperative  (Group)  Project,  24 
hours  before  the  scheduled  presentation. 

Working  With  Your  Group 

Your  group  meets  primarily  in  a designated  Florizon  Wimba  Group  Deliberation  Room.  Each  group 
meeting  must  be  archived  for  the  instructor's  future  references.  It  is  highly  recommended  that  you  follow 
the  Star  Legacy  model  and  meet  through  audio  conferencing  and  other  features,  like  application  sharing, 
in  Florizon  Wimba.  You  are  advised  to  start  this  planning  as  early  as  possible  and  do  your  part  first. 

Your  team  may  want  to  determine  who  is  responsible  for  what  information.  To  do  so,  you  need  to  take 
all  it  can  to  get  to  know  your  teammates.  After  knowing  each  other's  knowledge,  skills,  and  attitudes, 
you  as  a team  may  start  the  division  of  labor.  You  can  select  someone  to  act  as  moderator  for  the  group 
during  the  online  discussions  and  presentations.  One  way  of  presenting  the  group's  findings  is  to  assign 
individuals  to  be  responsible  for  determining  who  the  intended  audience  will  be,  what  resources  will  be 
required,  and  what  assessment  activities  will  be  necessary  to  conduct.  Everyone  in  the  group  can 
contribute  to  the  research  (conceptual  background)  as  well  as  diagnosing,  designing,  and  developing  the 
assessment  activities.  Each  individual  will  be  responsible  for  presenting  their  portion  of  the  work  during 
the  presentation.  Do  your  part  first  and  do  it  well. 

After  the  oral  presentation,  each  group  or  individuals  is  expected  to  define  and  / or  defend  group  work 
by  posting  revised  work  under  the  initial  posting  before  the  last  class  day  for  a better  grade. 
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Suggested  Presentation  Topics 

For  your  cooperative  project  you  may  choose  any  of  the  following  topics  or  the  like: 

[Practices] 

• Professional  development  [for  the  4th  grade  mathematics  teachers]  in  my  school:  A Dick  and 
Carey  Model 

• The  instructional  design  of  a concept-mapping  lesson  for  the  3rd  grade  teachers 

• An  after  school  program  on  reading  techniques:  Recommendations  for  practitioners 

• Design  of  job  aids  for  customer  representatives  in  a Fortune  1000  company 

• Adoption  of  a decision  support  system:  An  e-Learning  lesson  from  a multinational  company 

• Communication  training  for  Boeing's  X45  unmanned  jets'  operators/controllers  using  a virtual 
environment 

[Theories:  Models] 

• The  Kemp  Model 

• The  Gerlach  and  Ely  Model 

• The  Bergman  and  Moore  Model 

• The  ASSURE  Model 

• The  Interservice  Procedures  for  Instructional  Systems  Development  Model 

• The  Diamond  Model 

• The  4C-ID  Model 

• The  ARCS  Motivational  Model 

• Electronic  Performance  Support  Systems  (EPSS) 

• Human  Performance  Technology  (HPT) 

• d-Learning,  e-Learning,  and  m-Learning 


Figure  2.  The  Cooperative  Project  task  in  the  ISD  class. 

Both  activities  above  are  facilitated  using  Horizon  Wimba  Live  Classroom,  a conference  management 
system.  The  conference  management  system  is  integrated  into  the  Blackboard  system  (see  Figure  3). 


416 


Blackboard  Academic  Suite  Mozilla  Firefox 


Re  6*t  v«w  So  Bookmarks  loot  Help 

r * • s^|'i  EB  http:/Aittc.blad(board.com/webappsA)ortal/frames<tjsp?tab-coi^sesau1-S2Fbin%2Fcommon%2Fcourse.pl%3Fcourse.  v (_)  Go  v.  1 


B2  UT  TeleCampus 

* © *» 

Home  Help  Logout 

□©IE, 


Announcements 
Syllabus 
Calendar 
Sessions 
Projects/tU's 
Discussion  Board 
Horizon  Wimba 
Text  Chat 


<§)  Control  Panel 
($5)  Refresh 


SOUSSE  QSSHSS  > BSaftt  AN  !H7TOgUgMW.lfaBi  > » UVE  CLASSROOMS 

tml  List  of  Rooms  and  Archives 


Create  Room  | 


IV.iii.'H 

— "m^nua 

Classroom  IEDTC  6321.80  Fall  2ppp| 

Room 

ii* 

» 

Rvnow*  | 

Classroom  (EDTC  6321.80  Fall  2006)  - 08/222006  20  00 

Archive 

uA 

m 

R«<nov«  | 

Classroom  (EDTC  6321  80  Fall  2006)  - 08/28/2006  19  00 

Archive 

HA 

* 

R«mov«  | 

Classroom  (EDTC  6321.80  Fall  2006)  - 09/09/2006  12.22 

Archive 

HA 

m 

| 

Classroom  (EDTC  6321.80  Fall  2006)  - 09/18/2006  19  01 

Archive 

HA 

m 

R*mov«  | 

Group  1 Deliberation  Room  (EDTC  6321  80  FA  06) 

Room 

HA 

m 

R*mov«  | 

Group  1 Deliberation  Room  (EDTC  6321.80  FA  06)  - 09/1 1/2006  19:25 

A/chrve 

iiA 

* 

R«mov*  | 

Group  1 Deliberation  Room  (EDTC  6321  80  FA  06)  - 09/18/2006  19  57 

Archive 

HA 

m 

R«mov«  | 

Group  2 Deliberation  Room  (EDTC  6321  80  FA  06) 

Room 

HA 

i 

Ramcv#  ] 

Group  2 Deliberation  Room  (EDTC  6321.80  FA  06)  - 09/1 1/2006  20  52 

Archive 

HA 

if 

R«rtcv«  1 

Group  2 Deliberation  Room  (EDTC  6321  80  FA  06)  • 09/18/2006  19  38 

Archive 

HA 

f 

Rtmov*  | 

Group  3 Deliberation  Room  (EDTC  6321.80  FA  06) 

Room 

HA 

* 

R«-rs 

Group  4 Deliberation  Room  (EDTC  6321.80  FA  06) 

Room 

HA 

f 

R«t>oy*  1 

Group  4 Deliberation  Room  (EDTC  6321.80  FA  06)  - 09/11/2006  20  02 

Archive 

HA 

m 

| 

Group  4 Deliberation  Room  (EDTC  6321  80  FA  06)  • 09/18/2006  19  39 

Archive 

HA 

* 

R*t<oy«  | 

< > 
Done 


A SL'i 


Figure  3.  A screen  shot  that  illustrates  Horizon  Wimba  Live  Classroom  integrated  into  Blackboard. 

In  addition  to  Horizon  Wimba  Live  Classroom,  where  both  group  deliberation  rooms  and  the  main 
classroom  are  created,  students  also  use  the  Blackboard  Collaboration  feature  (also  known  as  text-based  chat), 
Blackboard  Discussion  Board  (i.e.,  the  forum),  Blackboard  Course  Email,  Skype,  and  telephone  for  communications 
purposes. 


Summary 

This  paper  described  how  faculty  members  use  the  Star  Legacy  model,  coupled  with  a suite  of  tools  that 
includes  a “cutting  edge”  CMS,  to  develop  instruction  for  online  students.  To  bridge  the  gulf  between  the  theory  and 
the  practice,  the  present  paper  attempts  to  exemplify  the  working  model  for  the  participants/readers  through  the 
developmental  process  using  specific  examples  from  two,  completely  online  courses  in  a Master’s  level  degree 
program.  This  paper  also  shows  how  integrating  the  VaNTH  design  model  into  a course  management  system,  as 
well  as  a powerful  communications  package  can  result  in  literally  a new  level  of  interactivity  and  learning  in  online 
courses. 
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Ethics  7000  Asks: 

Should  “Exploring  Our  ISD  Field”  Come  First,  Last,  or  at  all? 

Yedong  “Terry”  Tao 
University  of  Wisconsin 

Richard  Cornell 
University  of  Central  Florida 

Abstract 

In  1978,  the  graduate  instructional  systems  design  master’s  program  at  the  University  of  Central 
Florida  began.  The  faculty  required  that  the  first  course  for  students  literally  took  them  immediately  into 
the  field.  The  faculty  felt  that  waiting  until  a graduate  student  had  completed  two  years  of  a master’s 
program  to  really  explore  the  field  was  a breach  in  ethics.  By  the  time  classes  had  been  completed  it  was 
too  late  to  turn  around  and  exclaim,  “Oh,  but  this  is  not  the  field  I wanted!” 

Does  truth  in  advertising  extend  to  the  reality  of  what  the  field  is  like,  before  one  pays  thousands 
of  dollars  in  tuition,  books,  living  expenses,  and  untold  commuting  miles,  in  the  case  of  those  involved  in 
this  scenario?  Is  there  cause  for  litigation  in  that,  what  was  said,  failed  to  match  reality?  The  authors  have 
one  opinion;  the  audience  may  have  another.  What  do  you  think? 

What  follows  is  an  historical  overview  of  the  course  design;  the  competencies  developed  through 
the  DACUM  process  that  the  UCF  constituent  based  suggested  be  included  within  the  master’s  curriculum; 
an  example  of  the  course  syllabus  used  in  2002;  a listing  of  the  sites  visited  over  time  with  descriptions  of 
many  of  them;  and  finally,  examples  of  basic  instructional  technology  course  descriptions  as  found  in 
several  different  universities. 


Underlying  Theoretical  Bases 

Dewey,  Dale,  Gagne,  Jonassen,  and  others  have  argued  that  first-hand  experience  trumps  theory 
delivered,  sans  context.  Dewey  (1897)  wrote,  in  his  work,  My  Pedagogic  Creed,  Article  IV  - The  Nature  of 
Method: 

I believe  that  much  of  the  time  and  attention  now  given  to  the  preparation  and 
presentation  of  lessons  might  be  more  wisely  and  profitably  expended  in  training  the 
child's  power  of  imagery  and  in  seeing  to  it  that  he  was  continually  forming  definite, 
vivid,  and  growing  images  of  the  various  subjects  with  which  he  comes  in  contact  in  his 
experience  (p.  77-80). 

Given  this  belief,  it  was  felt  that  much  of  the  initial  course  in  the  UCF  ISD  master’s  program 
Survey  of  Instructional  Systems,  be  devoted  to  formulating  the  student’s  imagery  that  would  sustain  them 
through  subsequent  classes,  once  they  had  experienced  multiple  facets  of  the  field. 

Edgar  Dale  would  later  expand  the  notion  of  experience,  placing  it  into  the  context  of  a mediated 
continuum,  ranging  from  the  concrete  to  the  abstract.  He  began  his  discussion  by  first  calling  attention  to 
the  base  of  the  Cone: 

The  base  of  the  cone  represents  reality  itself  as  we  experience  it  at  first-hand.  It  is  the 
rich,  full-bodied  experience  that  is  the  bed-rock  of  all  education.  It  is  the  direct 
purposeful  experience  that  is  seen,  handled,  tasted,  felt,  touched,  smelled.  It  is  the 
unabridged  version  of  life  itself-tangible  experience,  which  we  commonly  refer  to  as 
“something  you  can  get  your  fingers  on”  “something  you  can  sink  your  teeth  into,”  etc. 

(Dale,  1946,  p.  38). 

Following  on  Dale’s  credo  of  placing  the  direct  experience  at  the  base  of  his  Cone,  the  course  we 
designed  was  purposefully  designed  to  provide  those  touches  of  reality  Dale  so  aptly  described. 

Gagne,  given  his  military  training  experience  in  the  1960’s,  took  two  decades  to  articulate  what  would 
become  the  mantra  of  instructional  designers  around  the  world  - his  nine  instructional  events 
recommended  for  successful  teaching  of  a specific  objective  (1985).  Considering  that  the  major  objective 
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of  the  Instructional  Systems  Survey  class  was  to  arrive  at  a career  decision,  the  course  design  made 
concerted  efforts  to  ensure  that,  throughout  the  course,  the  focus  reflected  those  events: 

Gain  attention  - In  this  class  you  will  experience  varieties  of  ISD  careers  by  means  of  visiting  a number  of 
different  organizations  who  create  or  provide  instructional  design  products  or  services. 

Identify  objective  - pose  question:  "Is  the  field  of  ISD  for  me?"  How  will  you  know  and  what  action  will 
you  take? 

Recall  prior  learning  - how  do  these  job  sites  compare  with  your  previous  work  experiences?  As 
we  progressively  explore  these  sites  can  you  apply  what  you  have  experienced  to  the  accumulation  of  facts, 
rules,  procedures,  attitudes,  values  and  connect  these  to  previous  site  visitations? 

Present  stimulus  - Through  the  site  visits  your  hosts  will  provide  multiple  and  contrasting  forms  of  stimuli 
that  will  allow  you  to  measure  levels  of  consistency  on  a weekly  basis. 

Guide  learning-  point  out  ‘‘‘eternal  verities”  about  the  field  as  seen  during  site  visits;  ask  questions 
of  those  at  the  sites  that  validate  or  repudiate  professed  realities  observed.  Monitor  student  acceptance  of 
information,  attitudes,  skills  and  values  to  ensure  the  “Hawthorne  Effect”  in  minimized. 

Elicit  performance  - ask  students  to  rank-order  their  site  preferences  and  tell  why,  indicating  what  ISD 
model  the  site  emulated.  Begin  to  assemble  group  project  segments  in  support  of  final  project  completion. 
Provide  feedback  - affirm  student  opinions,  discuss  alternative  views,  suggest  additional  attributes  for  the 
field  not  observed,  validate  conclusions,  encourage  students  to  posit  their  arguments  in  support  of  or 
against  conclusions  reached  about  specific  sites. 

Assess  performance-  provide  students  results  of  their  assessments  and  indicators  that  demonstrate 
the  student  is  arriving  at  a career  choice  decision. 

Enhance  retention/transfer  - through  group  projects  that  depict  sites  that  model  contemporary  ISD 
practices,  draw  conclusions  that  will  pertain  to  the  next  series  of  courses  that  were  derived  from  the  Survey 
course.  Encourage  students  to  make  individual  site  visits  to  organizations  of  specific  interest;  reinforce  that 
the  information  obtained  and  experiences  gained  in  this  course  are  designed  as  an  overview  and  the  the 
remaining  classes  will  ass  the  specific  skill  sets  needed  to  achieve  basic  performance  levels.  Encourage 
students  to  consider  applying  for  internships  at  more  than  one  type  of  site. 

Apologies  to  the  late  Robert  Gagne  for  taking  perfectly  good  instructional  events  and  morphing 
them  into  the  above  variants. 

Surround  students  with  reality  and  they  will  define  the  context  as  it  applies  to  them.  Exclude  the 
context  and  they  remain  detached.  With  a strong  cognitivist  emphasis  in  the  early  years  of  the  program 
(1978-1993),  other  methods  of  teaching  emerged  and  the  course  took  a decidedly-constmctivist  turn  in  its 
emphasis  and  subsequent  outcomes.  We  still  revered  the  theories  and  practices  of  the  aforementioned 
authors,  we  saw  wisdom  in  models  proffered  by  Keller,  Dick  and  Carey,  as  well  as  Heinich,  Molenda,  et  al. 
but  we  were,  at  least  in  the  aims  of  this  course,  tiying  to  prepare  the  students  for  a professional  life  of 
activities  based  on  collaborative  efforts. 

Jonassen  and  Rohrer-Murphy  (1999)  provided  new  insights  as  how  we  chose  to  amplify  the 
objectives  of  the  course.  They  suggested  activity  theory  as  an  alternative  basis  for  designing  of 
constructivist  learning  environments  (CLEs): 

Activity  theory  provides  an  appropriate  framework  for  analyzing  needs,  tasks,  and 
outcomes  far  designing  CLEs.  Activity  theory  is  a socio-cultural,  socio-historical  lens 
through  which  designers  can  analyze  human  activity  systems.  It  focuses  on  the 
interaction  of  human  activity  and  consciousness  within  its  relevant  environmental 
context.  Since  conscious  learning  emerges  from  activity  (performance),  not  as  a precursor 
to  it,  CLEs  should  attempt  to  replicate  the  activity  structures,  tools  and  sign  systems, 
socio-cultural  rules,  and  community  expectations  that  performers  must  accommodate 
while  acting  on  some  object  of  learning  (p . 6 1 ) . 

Perhaps  another  influence  of  moving  more  to  the  constructivist  side  of  the  scale  was  the  author’s 
own  continuing  evolution  to  a teaching  model  that  increasingly  involved  the  emphasis  on  collaboration, 
interpersonal  skill  development,  and  abiding  fascination  with  changing  media  formats,  all  of  which  would 
become  tools  of  these  master’s  students. 

During  this  same  period  the  UCF  Instructional  Systems  master’s  program  accepted  a number  of 
students  from  abroad,  many  of  them  from  Asia.  International  students,  especially  those  for  whom  this  was 
their  first  trip  to  the  United  States,  need  the  context  even  more,  not  only  to  extrapolate  what  is  learned  in 
America  to  home  country  (Cornell,  R.A.,  Pan,  S.,  and  Tsai,  M.  (April  2002)  but  in  addition,  to  justify  why 
they  pay  twice  or  more  the  cost  of  courses  taken  than  their  American  peers. 
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Given  these  premises,  when  should  students  begin  to  experience  the  field  of  their  choice?  We  hold  sooner  is 
better;  for  many  reasons.  The  first  is  the  epitome  of  practicality  - if  you  can  see  the  product  before  you  buy 
it,  you  have  the  opportunity  to  ask  the  seller  questions,  thus  ensuring  that  what  you  are  buying  is  what  you 
get. 

Put  another  way,  imagine  sitting  through  a year  and  a half  of  coursework  and  yet  to  put  foot  inside 
an  organization  that  designs  instruction.  When  that  day  arrives,  your  context  will  be  the  words  of  your 
professors,  the  writings  you  have  read,  and. . .yes,  please  go  on. . . 

Will  a final-semester  internship  compensate  for  the  deficit  of  not  having  first-hand  experience  or  insight 
about  the  field  ahead  of  time?  We  think  not.  Our  point  is  that  the  course  we  recommend  should  come  first 
in  a student ’s  program,  not  last!  In  doing  so,  we  have  sent  a strong  ethical  message  to  our  students. 

We  said  to  our  new  master’s  students,  here  is  our  field,  with  all  of  its  attributes,  its  excitement,  and  also,  its 
realities.  We  stated  up  front,  that  the  objective  of  the  course  is  for  the  student  to  make  an  informed  career 
decision  by  the  end  of  the  semester.  If,  at  the  end  of  the  course,  the  student  feels  this  is  not  the  field  for 
them,  great!  Think  of  all  that  tuition  money  saved  by  making  a decision  not  to  continue!  In  most  instances, 
we  find  that  students  completing  the  course  complete  the  master’s  program  and  many  then  apply  for  entry 
into  the  PhD. 

The  course,  then  entitled,  Survey  of  Instructional  Systems,  begins  by  providing  each  student  with  a 
list  of  competencies  they  will  eventually  acquire,  some  in  their  classes,  others  through  semester-long 
internships,  and  others  upon  being  hired  in  their  first  instructional  design  position.  See  attachment  1 for  the 
course  syllabus  and  attachment  2 for  the  competencies  list. 

Questions  to  Answer  When  Visiting  the  Sites 

Students  are  given  issues  to  investigate  and  questions  to  answer  as  they  visit  10-12  organizations. 
Depending  on  the  availability  of  corporate  sites  determines  how  many  visits  during  the  semester.  The  more 
the  better.  Areas  of  inquiry  involve  answers  to  the  following: 

1.  Here  are  the  places  we  work  (note  the  size  of  my  cubicle  or  work  station  vs.  that  of  my  colleague). 

2.  Here  is  how  I dress,  on  casual  days,  on  days  when  clients  come. 

3.  This  is  the  salary  I get,  within  company  parameters. 

4.  Here  is  the  amount  of  travel  asked  of  me. 

5.  These  are  advancement  possibilities,  including  pay  and  position. 

6.  This  is  how  I am  evaluated,  and  by  whom. 

7.  These  are  our  clients,  for  whom  we  design  instruction. 

8.  This  is  what  they  expect  from  me,  considering  that  my  degree  is  in with years  of 

experience. 

9.  These  are  the  tools  provided  me  to  do  my  work,  and  of  these were  used  in  class  when  I was  a 

student. 

10.  Here  are  typical  timelines  for  projects,  and  the  system  we  use  in  forecasting  task  accomplishment. 

11.  Here  is  the  number  of  times  we  met  the  timelines,  and  what  rewards,  if  any,  were  received  for 
doing  so. 

12.  Here  is  the  number  of  times  we  missed  the  timelines. 

13.  Here  are  some  reasons  why  we  missed  the  project  deadline. 

14.  Here  is  an  example  of  the  penalties  we  paid  because  we  missed  the  project  deadline. 

15.  This  is  my  benefits  package. 

16.  This  is  the  number  of  days  off  I get  each  year. 

17.  Here  are  the  sources  of  help  when  I need  them. 

18.  Here  are  the  colleagues  I work  with  and  why  they  are  important  to  me. 

19.  These  are  the  different  cultural  entities  with  whom  I work  on  a daily  basis. 

20.  Here  are  the  conceptual  ISD  models  we  employ. 

21.  Here  are  our  financials,  in  general  terms. 

22.  These  are  our  clients,  and  the  number  of  months  or  years  they  have  been  our  clients. 

23.  Here  are  the  kinds  of  projects  we  have  worked  on  this  year. 

24.  This  is  who  owns  the  company  and  here  is  where  they  are  located. 

25.  When  I graduated  from  the  program  I had  these  major  skills. 

26.  Here  is  what  I really  like  about  working  here. 

27.  Here  is  what  I wish  could  be  different  about  working  here. 
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28.  On  a scale  of  1-10,  with  10  being  tops,  my  happiness  in  working  here  is  a . 

29.  Here  are  some  things  I wished  I had  learned  when  I was  in  the  program. 

30.  Here  are  our  personnel  policies  related  to  equal  employment,  diversity,  gender  equity,  sexual 
orientation,  citizenship,  security  clearance  requirements,  pensions,  overtime,  tuition 
reimbursement,  maternity  leave,  bereavement  leave,  hiring,  release  from  position,  and  retirement. 

Logistics  Related  to  Course  Management 

In  the  late  70  ‘s  until  the  present,  the  need  for  instructional  designers  fluctuated  but  basically 
remained  consistent  in  an  upward  growth  mode.  Central  Florida  has  become  a hub  for  simulation,  training, 
and  more  recently,  instructional  gaming.  As  the  technologies  have  evolved,  so  too  has  the  need  for  more 
diversity  in  skill  sets.  In  the  late  70’s  into  the  mid-80’s,  the  emphasis  was  mostly  on  basic  ISD  skills.  With 
the  advent  of  eLearning,  however,  and  later  simulation  and  gaming  initiatives,  the  ISD  base  has  expanded 
to  include  a plethora  of  tangential  skills.  Within  this  context  the  expressed  need  for  a master’s  degree 
program  in  instructional  system  design  evolved.  It  was  decided  at  the  outset,  that  the  first  course  in  the 
program  would  be  one  that  provided  the  fundamentals  of  the  field  and  those  who  worked  within  it. 

For  this  kind  of  program  to  be  successful  it  had  to  depend  upon  the  good  will  of  the  organizations 
who  would  hire  its  graduates.  Our  plan  was  to  take  the  students  to  the  workplace  - the  sites  to  be  visited. 
The  organizations  were  told  that  the  basic  class  was  made  up  of  students  whom,  for  the  most  part,  had  little 
or  no  previous  ISD  experience.  Given  that  the  initial  group  of  organizations  had  participated  in  our  first 
DACUM  analysis,  they  had  already  told  us  what  workplace  skills  they  were  seeking. 

The  DACUM  process  (Developing  A Curriculum)  (n.d)  was  employed  in  1978  to  “identify  instructional 
needs,  instructional  program  planning,  curriculum  development,  training  materials  development,  creating 
and  revising  job  descriptions  and  standards,  employee  recruitment,  ISO  9000/1400  performance,  and  career 
guidance”  (http:www.ateec.org/curric/dacum/dacum_overview.htm). 

The  class  traditionally  met  once  each  week,  in  the  evenings.  During  fourteen  weeks,  four  required 
attendance  on  campus,  with  the  remaining  ten  classes  being  held  at  a difference  site  each  week. 

The  first  task  was  to  find  ten  organizations  that  would  be  willing  to  keep  their  key  staff  at  work  for  two-to- 
three  hours  after  the  workday  ended.  Initially  this  was  a hard  sell,  at  least  for  the  first  year  or  two.  It  then 
became  apparent  to  the  organizations  that  opening  their  doors  to  the  class,  many  of  whom  might  become 
potential  employees,  made  good  business  sense.  Once  it  dawned  on  the  site  owners/managers,  that  their 
quest  to  locate  new  hires  was  solved,  at  least  at  the  entry  level,  finding  willing  organizations  to  host  the 
class  each  year  ceased  to  be  a problem. 

Initially  there  were  few  students  in  any  of  the  organizations  we  chose  to  visit.  As  time  passed, 
however,  increasing  numbers  of  the  UCF  ISD  graduates  were  employed  locally,  thus  affording  a valuable 
contact  within  who  often  served  as  catalyst  for  the  company  opening  their  doors  after  hours. 

An  aside.  Either  through  serendipity  of  just  dumb  luck,  one  of  the  first  organizations  to  host  the  class  was 
the  Darden  Restaurant  in  its  new  training  center  based  in  South  Orlando.  Upon  our  arrival  at  the  training 
center,  then  designed  to  train  employees  in  the  new  Red  Lobster  brand,  the  class  was  welcomed  by  training 
lead,  Sam  Jones,  and  ushered  into  a room  where  a sumptuous  seafood  buffet  awaited  the  class! 

Word  of  the  Red  Lobster  generosity  spread  quickly  through  the  other  companies  and  organizations 
and  almost  immediately  it  became  de  rigueur  for  the  sites  to  begin  their  presentations  with  an 
accompanying  series  of  culinary  offerings,  ranging  from  sodas  and  cookies  to  full  buffets.  This  was 
extremely  helpful  to  our  students  considering  most  of  them  worked  full  time  and  attending  the  evening 
class  precluded  them  stopping  off  somewhere  to  get  a quick  snack  before  class.  {Author ’s  note : It  is 
speculated  that  some  of  the  sties  subsequently  evaluated  by  the  class  suffered  reduced  evaluations  by  their 
not  meeting  the  catering  criteria  of  other  organizations.) 

EME  5054  Sans  MapQuest 

Once  we  were  able  to  line  up  site  visits  (some  were  announced  later  in  the  semester  pending 
corporate  approvals)  the  next  step  was  to  provide  the  students  with  directions  to  the  site.  Given  that  most 
were  commuting,  the  directions  had  to  be  written  from  multiple  points  on  the  map.  Those  sites  in  the 
Research  Park  adjoining  UCF  required  minimal  guidance;  those  that  were  in  other  parts  of  Orlando,  were  in 
adjoining  counties,  or  located  several  hours  distant  (and  there  were  some  of  these),  required  considerably 
sophisticated  linguistic  and  cartographic  skills  by  the  instructor  and  the  ability  to  follow  directions  by  the 
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students.  Students  soon  learned  to  ask  their  peers  for  help  if  they  encountered  problems  locating  the  site. 
Car-pooling  solved  many  location  identification  issues  but  not  all.  The  advent  of  MapQuest  eventually 
solved  the  geography  problem,  for  most. 


Extolling  Virtuosity 

On  a number  of  occasions  students  arrived  late  to  a site  or  appear  dressed  in  less-than-acceptable 
clothing.  In  addressing  the  first  issue,  it  became  a rule  that  any  student  arriving  more  than  two  minutes  late 
to  a sire  would  find  that  site’s  door  locked.  One  time  a carload  of  six  students  arrived  late,  only  to  find  they 
were  locked  out.  This  proved  an  especially  painful  experience  as  the  site  was  sixty  miles  distant  from  the 
university.  After  that  experience,  being  late  to  a site  was  very  rare. 

With  the  advent  of  cell  phones,  now  ubiquitous,  students  in  the  mid-90’s  onward  at  least  had  the 
option  of  calling  me  at  the  site,  should  an  event  beyond  their  control  arise. 

The  matter  of  “acceptable”  dress  arose  with  every  class  year  and  time  was  spent  addressing  grooming, 
clothing  choice,  and  interpersonal  communications.  While  one  might  think  such  issues  would  not  be 
necessary  with  graduate  students,  they  were  for  some.  Eventually,  peer  modeling  of  behaviors  and  dress 
became  the  norm  and  these  concerns  gradually  dissipated. 

Pros  and  Cons  of  Managing  the  Course 

The  benefits  of  having  the  Instructional  Systems  Survey  class  at  the  beginning  of  the  master’s 
program  include: 

• If,  at  the  end  of  the  course,  the  student  indicates  that  ISD  is  not  the  field  for  them,  they  have  saved 
thousands  of  dollars  that  can  then  be  used  for  entry  into  another  program  or  used  for  other 
purposes. 

• After  having  observed  many  of  the  students  in  the  entry  level  course,  managers  at  many  of  the 
sites  follow  up  with  specific  inquiries  as  to  who  a given  student  was  and  how  they  might  invite 
them  to  come  for  an  interview.  Many  students  have  been  hired  on  the  basis  of  their  demeanor  and 
demonstrated  intellect  noted  during  attendance  at  one  of  the  site  visits. 

• The  student  who  does  make  a decision  to  enter  the  ISD  field  has  a clear  understanding  of  the 
innumerable  variations  found  in  companies  and  organizations  that  create  ISD-based  products  and 
services.  Consequently,  after  having  reviewed  the  competencies  aligned  with  the  program,  they  do 
self-assessments  quickly,  choosing  to  opt  for  classes  wherein  they  may  lack  strength  in  specific 
areas  they  deem  important  to  the  job  they  seek. 

• Over  time,  a cadre  of  very  strong  supporters  in  companies  and  organizations  emerges.  Some  of 
their  enthusiastic  support  stems  from  the  fact  they  now  have  a number  of  UCF  graduates  working 
for  them.  Other  constituents  see  the  strong  relationships  that  emerge  between  their  expressed 
career  skills  and  the  skills  brought  them  from  the  program. 

• By  the  completion  of  the  ISD  masters,  the  Survey  course  is  the  one  most  often  cited  as  being  their 
“best  class.”  The  instructor  assigns  this  credit  away  from  him  but  to  the  excellence  found  in  these 
organizations  and  how  the  students  perceived  them. 

• Because  of  the  nature  of  the  course,  with  its  ever-changing  venues,  there  is  a refreshing  newness  to 
each  class.  It  was  necessary  to  vary  the  types  of  sites  chosen  to  ensure  that  this  freshness  was 
maintained. 

• Those  areas  that  might  be  termed  negatives  include: 

• To  the  teacher-centered  professor,  relinquishing  all  that  valuable  face-time  to  others  was  viewed  as 
their  being  vulnerable.  In  a course  such  as  this,  the  instructor  is  truly  a facilitator;  ergo,  the 
professor  who  needs  the  constant  control  of  his  or  her  class  will  not  be  comfortable  with  such  an 
arrangement. 

• To  set  up  the  course  logistics  each  year  requires  an  excessive  amount  of  work,  much  of  it  requiring 
that  the  instructor  maintain  continual  good  relationships  with  the  companies  and  organizations 
within  the  university  service  areas  and  beyond. 

• The  student  who  has  a high  need  for  individual  recognition  will  find  a course  such  as  this 
restrictive  in  that  their  grade  is  primarily  based  on  their  ability  to  work  well  in  a teamed 
environment. 
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• The  concept  of  self-motivation  is  an  underlying  principle  for  the  program  as  are  interpersonal 
skills.  If  student  lack  motivation  or  “people”  skills,  they  will  not  be  happy  in  this  program. 

• Sometimes  the  expectation  of  those  at  the  various  sites  exceeds  the  capacity  of  the  students,  given 
that:  a.  this  is  their  first  course  in  the  program,  b.  students  from  all  bachelor’s  degree  majors  are 
accepted  into  the  program. 

• Students  struggling  financially  may  encounter  initial  difficulties  in  traveling  to  sites  or  they  may 
not  own  much  appropriate  clothing.  Eventually  the  students  resolve  these  issues,  sometimes  with 
the  help  of  peers  or  from  the  instructor. 

Conclusions 

There  is  no  doubt  that  the  course,  Survey  of  Instructional  Systems  served  its  purpose.  Of  the 
almost- 1,000  students  who  enrolled  in  the  program,  ten  decided  that  ISD  was  not  for  them.  Probably  of  the 
1,000+  who  chose  to  remain,  twenty-five  failed  to  complete  the  master’s.  Those  odds  speak  for  themselves. 
If  the  author  had  it  to  do  again,  he  would  find  a young  professor  filled  with  fire  in  his  or  her  belly  and 
assign  them  this  class.  He  or  she  would  be  a person  the  students  are  excited  about  being  around,  who  was 
genuine,  self-effacing,  filled  with  humor  and  caring  for  all  of  the  students.  Author’s  Note:  The  course  as 
described  is  no  longer  taught. 
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What  does  "interpretation"  have  to  do  with  instructional  design,  and 
why  on  earth  would  we  need  an  "anti-interpretation"  instructional 

design  methodology? 

Ray  Taylor 
Dr.  Johannes  Strobel 
Concordia  University,  Montreal  QC 

The  short  answer  is  simple.  By  the  time  a learner  is  exposed  to  "designed"  instruction,  the  steps  taken  in  the 
production  of  learning  content  have  interpreted  and  re-interpreted  data  so  many  times  as  to  render  it 
ambiguous.  The  consequence  is  that  more  time  is  spent  in  various  interpretations  of  content  as  to  render  it 
extremely  difficult  to  recontextualize  by  the  learner.  The  intuition  is  that  excessive  time  spent  in 
interpretive  activities  is  interfering  with  the  learning  process,  therefore  an  anti-interpretive  instructional 
design  methodology  might  improve  learning  performance. 

To  the  academic  reader  such  as  yourself,  for  which  this  report  is  intended,  has  been  trained  in  a culture  that 
promotes  and  uses  analytic  methodologies  for  the  presentation  of  arguments  and  statements  about  subject 
matter.  Your  automatic  (or  semi-automatic)  response  to  reading  the  above  paragraph  is  most  likely  to 
question  or  to  try  and  map  out  the  assumptions  and  for  the  more  enlightened,  beliefs  and  positions  about 
learning  theories,  philosophy  (perhaps)  or  even  the  scientific  enterprise  of  empiricism.  These  are  among  the 
hallmarks  of  analytic  techniques,  and  you  would  rightly  expect  that  some  statements  about  things  like 
"interpretation"  and  "instructional  design"  would  be  illustrated  in  an  analytical  way.  This  is  usually  in  the 
form  of  a series  of  arguments  or  some  empirical  evidence  that  justifies,  and/or  shows  logically  why  we 
should  be  paying  attention  to  this  sort  of  thing1. 

The  preceding  paragraph  is  intentionally  crafted  to  draw  a parallel  to  "designed"  instruction  that  has  been 
analytically  derived.  Like  the  first  paragraph  of  this  report,  most  designed  instruction  tend  to  deliver 
summary  information  that  has  been  pre-interpreted  and  condensed  in  such  a way  that  leaves  the  learner 
without  much  grounding  with  which  to  provide  adequate  context.  In  other  words  inadequate  justification  of 
content  by  way  of  contextual  or  external  referencing  is  naturally  ambiguous.  And  here  is  the  non-essential 
problem:  even  though  we  can  consider  the  average  analytically  trained  person  reading  this  report  to  have  a 
reasonable  amount  of  cognitive  skills  to  use  a variety  of  theoretical  groundings,  without  adequate  context, 
she  flounders  in  the  ambiguity  of  multiple  potential  interpretations.  I could  present  the  first  paragraph  at  a 
round  table  discussion  and  hours,  days  or  even  months  could  be  spent  in  exploring  possible  explanations  by 
stating  theoretical  frameworks  and  processing  those  four  meager  sentences  and  interpreting  them  as  to  have 
some  kind  of  "meaning"  within  a chosen  theoretical  framework.  This  generally  includes  other  non-essential 
arguments  about  the  appropriateness  of  a favored  interpretive  model  (which  I will  use  interchangeably  with 
the  term  "theory")  as  the  analytically-trained  are  prone  to  do.  This  is  descriptive  of  the  enterprise  of 
academia,  and  it  is  also  the  enterprise  of  instructional  design.  This  critique  examines  in  an  analytical  way, 
the  consequences  of  the  analytical  approach  to  instructional  design  as  an  interpretive  process  mechanism , 
and  presents  the  idea  of  a more  natural  anti-interpretive  approach  to  designing  instruction. 

As  I mentioned  "non-essential  arguments  about  theoretical  models"  I will  attempt  to  clarify  with  a self- 
referential  comparison  to  typical  analytic  writing.  At  this  point  in  an  academic  report  or  article  some  sort 
of  ideological  statement  is  made,  or  at  least  a clue  is  given  to  the  author's  theoretical  allegiance,  often  even 
stated  in  the  title  of  the  article.  This  is  a strategy  to  disambiguate  the  statements  that  appear  in  the  thesis  or 
in  the  supporting  evidence.  For  example,  if  I were  to  state  that  this  is  a "behaviorist",  "social- 
constructivist",  "cognitivist"  (or  whatever)  allegiance,  I am  actually  providing  a directive  on  how  to 
interpret  the  text  by  way  of  the  assumptions  of  the  descriptive  theory.  It  is  shorthand  for  preceding  each 
proposition  of  the  article,  as  justified  by  "factual  evidence"  or  by  its  place  in  a philosophical-dialectical 


1 The  full  stated  assumptions  and  discussion  of  theoretical  frameworks  and  justification  for  an  anti- interpretation 
instructional  design  methodology  will  be  in  the  author's  forthcoming  M.A.  Thesis,  tentatively  entitled.  Interpretation, 
Anti-Interpretation  and  ADDIE. 
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discussion  with  "if  the  assumptions  of  theory  X are  true,  then  these  propositions  are  true."  If  an  ideological 
statement  or  a proposed  interpretive  framework  is  absent,  astute  academic  readers  would  automatically 
assume  that  the  report  is  to  be  interpreted  according  to  my  previously  known  ideological  positions.2  The 
instinctual  intent  in  this  strategy  is  precisely  to  avoid  the  kind  of  non-essential  judgments  and  critiques 
about  positions  that  are  political  in  nature,  in  other  words,  about  having  justifiable  beliefs  for  being  in  one 
ideological  camp  or  another.  However  we  make  a huge  assumption  that  despite  interpretive  directives,  the 
content  will  be  interpreted  according  to  the  author's  intent. 

Since  I am  not  well  known  and  I have  not  stated  an  ideological  position,  the  analytically  trained  reader 
might  assume  that  this  is  a meta-theoretical  piece  as  it  questions  theoretical  assumptions  of  theories,  or 
more  specifically  the  assumptions  of  theoretical  analysis  (this  is  partially  correct).  To  provide  some 
additional  grounding,  anti-interpretation  is  not  about  radicalizing  an  atheoretical  or  anarchic  (Feyerabend, 
1988)  or  synthetic  (Spencer,  1895)  approach  , but  that  purely  analytical  approaches  to  designing  instruction 
is  reductive  in  the  sense  that  too  much  source  data  is  ignored,  and  therefore  instrumentally  misapplied  to 
the  objects  or  units  of  analysis. 

The  analytical  approach  to  "meaning-making"  is  really  no  different  for  any  learner  presented  with 
"designed"  content,  in  the  sense  we  promote  analytical  and  critical  thinking  skills  from  about  the  age  of 
puberty  as  part  of  our  mandatory  systems  of  education,  and  somehow  we  assume  that  an  analytical 
technique  is  the  best  way  to  decode  instructional  content.  It  is  also  obvious  that  those  who  manage  to  do  so 
successfully  usually  go  on  to  become  professional  analysts  in  either  business  or  academia.  The  intuitive 
strategy  is  to  disambiguate  in  a "scientific"  way  a mass  of  discretely  (ontologically)  classified  data  to  make 
it  meaningful.  Meaningfulness  is  defined  here  as  what  follows  after  description,  in  other  words,  it  is  the 
result  of  processing  data  using  an  interpretive  framework  as  an  instrument  of  context.  The  consequence  of 
this  approach  is  that  meaning  is  not  relative  to  the  source  data  or  the  "real  world"  phenomena,  but  rather  to 
the  interpretive  framework  or  the  theory.  In  other  words,  it  is  not  experiential  (observable)  phenomena  that 
are  linked  to  the  theoretical  interpretation  but  the  ontologically  derived  assumptions  of  data  descriptions.3 

There  are  other  disambiguation  strategies,  but  broadly  speaking  the  analytical  approaches  in  designing 
instruction  are  audience  or  learner  analysis,  context  or  environmental  analysis,  and/or  goal  and  task 
analysis  (Jonassen,  Tessmer,  & Hannum,  1999).  These  are  all  supposed  to  give  the  instructional  designer  an 
interpretive  framework  in  which  to  evaluate  "activity"  or  "content"  in  such  a way  that  it  may  be  presented 
in  the  most  efficient  way  possible.  This  is  no  different  than  your  typical  academic  report,  whereby 
ideological  statements,  presentation  of  evidence  and  even  logical  arguments  are  all  used  in  a way  that 
assume  the  readers  have  the  analytical  skills  to  process  the  information  and  create  "meaning"  through 
selective  omission.  But  this  is  no  guarantee  that  the  content  is  (extrinsically)  valid,  as  a skeptical  academic 
will  believe  or  not  believe  the  presentation  of  the  content  largely  based  on  his  own  ideological  beliefs.  For 
example,  I might  say  that  this  report  is  a call  to  a return  to  logical  positivism  and  Skinnerian  behaviorism, 
which  despite  the  evidence  I present,  would  be  dismissed  as  outmoded  by  anyone  who  adheres  to  an 
ideological  position  that  says  that  we  are  not  justified  in  having  those  beliefs  (which  is  the  current  fashion 
in  learning  theory). 

This  is  the  dilemma  facing  the  instructional  designer.  In  that  through  distillation  of  content  and  presenting 
it,  he  cannot  possibly  anticipate  how  it  will  be  judged  against  the  learner's  own  beliefs,  whether  those 
beliefs  are  experientially,  culturally,  socially  or  academically  derived.  The  problematic  is  that  we  can 
analyze  to  death  all  of  the  above-mentioned  operating  parameters  of  the  learning  process,  but  as  long  as 
content  is  being  presented,  it  will  be  a question  of  rhetoric  or  poetics  as  to  whether  or  not  the  learner's 
beliefs  will  be  sufficiently  in  agreement,  or  altered  (the  case  may  be).  The  astute  reader  at  this  point  might 
be  thinking  that  this  is  a "communication"  or  "message  design"  view  of  learning,  and  this  is  not  what 
learning  is  all  about.  If  this  is  true,  then  we  must  to  let  go  of  analytically-motivated  and  derived  data  as  the 
primary  mode  of  representation  (and  I will  expand  on  this  notion  further  on  in  the  report).  The  underlying 
assumption  of  the  analytical  approach  says  we  are  justified  in  presenting  the  "essence"  of  subject  matter 
that  will  be  somehow  easier  to  understand  by  learners.  By  taking  the  analytical  approach,  I venture  that  we 


2 1 have  the  luxury  of  being  unknown,  and  I am  using  this  to  my  advantage. 

3 This  view  is  a summary  of  post-structuralism  as  represented  by  Gadamer,  Foucault,  Derrida  and  Levi-Strauss. 


426 


are  simply  deluding  ourselves  with  analyses  that  we  believe  should  be  sufficient  justification  for  the  content 
presentations  we  come  up  with. 

The  problem  is  that  WE  believe  our  conclusions  and  content  choices,  but  the  learner,  alas,  did  not  have 
access  to  the  analytical  process  that  formed  our  beliefs  about  the  content  and  our  choices  of  presentation. 
WE  (the  instructional  designers)  are  justified  in  our  beliefs,  but  somehow  our  justification  is  not  enough  for 
a learner  to  automatically  believe  what  is  being  presented.  To  demonstrate  this,  ask  yourself,  now  that  you 
have  read  just  a bit  more  than  1200  words,  if  you  really  have  enough  information  to  believe  the  four 
sentences  of  the  first  paragraph.  It  is  more  than  likely,  since  I have  intentionally  left  out  the  guideposts,  that 
you  don't  know  which  ideological  hat  to  wear  (or  which  one  I am  wearing),  or  what  I might  believe  or  what 
your  own  beliefs  about  instructional  design  or  learning  can  help  in  the  interpretation  of  those  statements. 

If  this  is  difficult  for  an  academic  audience,  imagine  what  it  must  be  like  for  non-professional  analysts,  in 
other  words,  for  a typical  learner,  when  faced  with  just  about  any  "designed"  instruction.  Just  because  an 
instructional  designer  has  verified  and  justified  beliefs  about  the  subject  material  is  not  any  reason  to  expect 
that  a learner  will  automatically  accept,  believe,  understand,  retain  and  use  it  in  a way  that  would 
demonstrate  that  it  is  "learned".  In  other  words,  there  are  not  enough  contexts  to  understand  what  I mean, 
not  enough  context  to  agree  or  disagree,  judge  or  remain  open,  believe  or  disbelieve  or  to  process  further  in 
order  to  arrive  at  a position  of  judgment  about  my  statements.  However,  by  not  being  careful  about  context, 
the  effect  is  more  likely  judgment  than  openness.  The  selection  of  context,  regardless  of  the  analytical 
approach  that  seeks  verification,  is  a crap-shoot  of  possible  meanings  when  it  reaches  the  learner4. 

Up  to  now  the  strategy  has  been  to  provide  context,  and  that  our  analytical  methods  intend  to  process  data 
into  that  context  as  a disambiguation  AND  a justification  strategy.  We  can  use  external  references, 
theoretical  frameworks,  the  productions  of  theoretical  frameworks  (logico-empirical  or  otherwise)  or 
experience.  These  are  all  correspondence  models  of  justification  and  they  simply  provide  the  background 
for  having  belief  about  certain  propositions.  This  is  just  to  say  that  any  statement  or  presentation  of  content 
is  conditional  and  dependent  upon  a good  deal  of  external  references,  theoretical  frameworks,  the 
productions  of  theoretical  frameworks  (logico-empirical  or  otherwise)  and  experience  (which  may  internal 
to  the  learner  or  external  by  what  has  been  learned  by  other  people),  stated  or  otherwise  unstated.  The 
intuition  is  that  we  can  be  increasingly  precise  in  our  content  and  context  descriptions  that  will  somehow 
facilitate  transfer  either  into  or  out  of  the  learner.  There  is  simply  never  enough  context,  but  can  we  assume 
that  a theoretical  framework  or  a belief  about  something  is  indeed  adequate  context? 

"The  short  answer  is  simple. " 

This  statement  represents,  to  a great  degree,  a promise  that  the  reader  will  be  able  to  apprehend,  grasp, 
"grok",  understand  or  get  the  gestalt,  meaning  or  insight  into  what  I am  proposing,  without  too  much 
difficulty  (notice  that  I used  seven  different  words  that  mean  roughly  the  same  thing).  But  it  is  fairly  easy  to 
attest  that  the  simplicity  or  ease  in  which  that  might  happen  is  largely  dependent  on  the  double-edged 
sword  of  our  abilities  as  communicators  within  the  academic  context.  This  is  to  mean  there  is  some  kind  of 
congruency  between  my  ability  as  a poet/rhetorician/  scientist  and  your  ability  as  a trained  analyst  to  accept 
or  reject  a particular  viewpoint  or  belief  about  something  we  might  have  a conversation  about,  in  this  case 
instructional  design  or  any  of  the  learning  theories  that  inform  it.  In  other  words,  the  difficulty  here  is,  once 
the  rudimentary  grammars  of  communication  and  language  have  been  overcome,  the  validity  of  the 
statement  "the  short  answer  is  simple"  is  relative  to  a very  large  web  of  beliefs  (Quine,  & Ullian,  1978), 
therefore  subject  to  interpretive  judgment.  We  might  agree  at  this  point  that  we  can  extend  this  to  the  "short 
answer  is  simple,  but  the  long  answer  is  not." 

The  promise  of  instructional  design  is  very  similar.  The  underlying  belief  is  that  by  appropriate  analytical 
techniques,  we  can  somehow  organize  knowledge  and  skills  in  such  a way  that  renders  it  simple  to 


4 Keep  in  mind  this  argument  depends  on  the  verifiability  of  experience  that  follows  a certain  internal  logic  and  not  one 
that  depends  on  some  kind  of  external  correspondence.  This  is  counter-intuitive  for  instructional  designers  as  the 
intuition  is  to  provide  context,  or  the  "right"  context,  given  the  results  of  analyzing  the  operational  assumptions  of 
learning  a given  content  or  activity.  The  disambiguation  strategy  here  is  to  be  consciously  ambiguous,  which  is 
different  that  presenting  ambiguity  or  more  precisely,  unintentionally  creating  ambiguity. 
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apprehend  ("understand",  or  what  have  you).  The  long  answer,  which  is  the  one  that  takes  place  when  the 
content  reaches  the  learner,  is  that  it  is  not  so  simple.  So  what  is  the  promise  of  instructional  design,  if  not 
to  compress  what  we  know  about  knowledge  and  skill  production  into  tightly  essential  packages  that  appear 
to  be  simple  by  nature  of  their  brevity?  Only  that  the  consequence  of  unpacking  knowledge  or  activities  in 
the  learning  context,  is  that  they  expand  so  much  and  so  quickly  that  we  don't  have  the  equipment 
(frameworks,  theories,  properly  trained  teachers,  etc.)  to  know  how  to  deal  with  the  decompression 
effectively5.  In  other  words,  instructional  design  is  largely  about  packaging  and  package  delivery  and  not 
really  about  what  happens  once  the  packages  are  delivered  to  the  warehouse,  that  black  box  we  refer  to  as 
the  learner's  mind.6 

"By  the  time  a learner  is  exposed  to  "designed"  instruction,  the  steps  taken  in  the  production  of  learning 
content  have  interpreted  and  re-interpreted  data  so  many  times  as  to  render  it  ambiguous.  " 

The  process  of  instructional  design  is  essentially  to  accomplish  one  of  two  things.  The  first  is  to  take 
previously  rationalized  knowledge  and/or  behaviors,  strip  them  of  their  original  conditional-contextual 
links  and  recontextualize  the  resulting  package  to  the  learner.  The  second  is  to  strip  the  learner  of  her 
context-conditioning  and  recontextualize  her  to  the  knowledge  and/or  behavior  "package".  Both  approaches 
require  extensive  hermeneutical  interpretation  on  the  part  of  the  learner.  It  should  be  possible  to  organize 
instruction  to  deal  with  the  interpretive  variables  that  are  encountered  in  the  learning  process,  rather  than 
assume  that  a prescribed  or  suggested  analytical  interpretation  will  be  used  by  the  learner. 

To  step  back  from  the  mechanical  picture  of  instructional  design,  the  more  abstract  notion  of  interpretation, 
or  the  role  of  interpretation,  can  be  roughly  aligned  to  the  discipline  of  hermeneutics.  I will  qualify  and  use 
this  term  not  by  its  romantic  origins  and  relationship  with  religion  but  rather  as  the  stance  one  takes  when 
encountering  data  or  phenomena  that  is  unknown,  unclassifiable  or  otherwise  inconsistent  with  either 
theory  or  experience  (in  other  words,  difficult  to  explain).  (Rorty,  1979)  This  stance  is  similar  to  the  radical 
hermeneutics  of  Derrida  and  Foucault  as  defined  by  (Gallagher,  1992)  7 . The  hermeneutical  state  is  the 
natural  state  of  learners,  and  not  the  analytical  one. 

If  we  consider  that  the  analytical  processes  of  instructional  design  are  also  largely  interpretive,  the  only 
possible  outcome  is  that  we  end  up  replicating  and  compounding  interpretations  of  interpretations8.  It 
would  be  sufficient  only  to  acknowledge  that  this  indeed  is  an  effect  of  the  process  of  instructional  design 
on  the  learning  context  and  worthy  of  study.  A deeper  examination  would  identify  specific  points  in  the 
process  where  an  analytical  interpretation  is  used  instrumentally,  that  is  transduced  into  another  subprocess 
of  instructional  design.  For  example,  the  results  of  a needs  assessment  is  transduced  into  a learning  design 


8 An  example  is  the  Calculus.  There  are  two  only  two  central  precepts,  differentiation  and  integration,  and  the  modeling 
equations  are  relatively  straight-forward.  However  it  takes  roughly  12  to  13  years  of  mathematics  instruction  before  it 
is  introduced  to  senior  high-school  or  junior  college  students. 

6 It  might  be  appropriate  here  to  question  whether  or  not  packaging  and  package  delivery  is  a good  representational 
metaphor,  which  is  another  often  abused  and  potentially  ambiguous  disambiguation  strategy.  If  we  object  to  this 
metaphor,  we  must  also  question  whether  we  should  use  an  objectified  model  of  knowledge  (concepts,  procedures,  or 
whatever  representation  is  appropriate  for  a given  "natural"  epistemology)  and  assume  that  these  bundles  of  objects  are 
somehow  transferred  and  accumulated  in  the  mind.  I would  like  to  take  this  further  and  say  that  the  notion  of  mind  as  a 
repository  of  knowledge-representations  is  also  a huge  assumption,  and  that  it  may  be  an  underlying  assumption  of  the 
data-compression  model  of  knowledge  I am  illustrating  here.  We  might  not  have  any  justification  at  all  to  have  these 
beliefs.  This  has  serious  consequences  for  the  use  of  analytical  techniques  to  arrive  at  the  kind  of  compressed  content 
data-descriptions  we  inadvertently  establish  as  a goal  of  well-designed  instruction. 

7 1 am  skipping  over  the  relationship  between  hermeneutics  and  metaphysics  however  it  is  worth  noting  that  the 
grounding  of  most  learning  theories  in  positivist  psychology  rejects  metaphysics,  hence  hermeneutics.  In  my  thesis  I 
expand  on  the  neo-pragmatist  idea  that  empiricism  in  the  natural  and  human  sciences  makes  huge  metaphysical 
assumptions. 

8 This  is  similar  to,  but  sufficiently  different  from  Gidden's  concept  of  the  "double  hermeneutic"  in  interpretive  social 
sciences.  In  The  Consequences  of  Modernity  Giddens  show  how  sociological  knowledge  becomes  part  of  the  social 
fabric,  hence  influences  and  modifies  the  interpretive  framework  that  created  the  knowledge  in  the  first  place.  My 
distinction  is  not  that  the  learning  process  is  self-referential,  as  social  agency  is  intentionally  limited  (usually)  to  the 
learning  context.  In  other  words,  the  people  in  a decontextualized  learning  environment  do  not  have  direct  influence  on 
the  social  sphere  or  community  of  practice(Wenger,  1998). 
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plan,  then  packaged  as  a result  of  developmental  and  implementation  constraints.  The  resulting  "learning" 
(if  you  could  call  it  that)  is  also  often  constrained  by  assessments  that  call  for  positivist  evidence  (visible 
and  measurable  phenomena). 

The  effects  of  designing  instruction  are  one  of  analyzing  data  for  the  purpose  of  transformation  for  the 
needs  of  the  process,  rather  than  the  learner.  As  stated  previously  one  consequence  are  the  interpretation  of 
the  interpreted  data,  rather  than  the  re-interpretation  of  source  data  in  each  of  the  ADDIE  sub-processes. 
Another  consequence  is  the  obscuring  of  the  extrinsic  goals  motivating  the  selection  of  content  and 
activities.  This  will  require  revisiting  some  notions  of  analytical  interpretation  as  they  are  formed  by 
learning  theories  and  extrinsic  goals. 

"The  consequence  of  an  analytical  instructional  design  process  is  that  more  time  is  spent  in  various 
interpretations  of  content  as  to  render  it  extremely  difficult  to  recontextualize  by  the  learner.  " 

Qualifying  this  statement  requires  a deconstruction  of  theoretical  models  of  education,  learning  and  the 
process  of  instructional  design,  which  is  presented  here  as  a three  part  strategy:  exposing  learning  theories 
as  interpretive  frameworks,  demonstrating  how  learning  theories  produce  deficiency  interpretation  models 
and  finally  how  extrinsic  goals  become  obscured  by  analytical  instructional  design  processes. 

Analytical  Interpretation  and  Learning  Theories 

The  proliferation  and  scope  of  learning  theories  in  general,  and  instructional  methodologies  in  particular,  is 
a healthy  thing  from  a philosophical  perspective.  It  is  an  indication  that,  as  educators,  we  show  a 
willingness  to  question  underlying  assumptions  and  beliefs  about  learning  and  teaching.  From  an  analytical 
perspective,  the  domain  of  education  has  virtually  exhausted  the  exploration  of  different  units  of  analysis, 
both  categorical  and  intrinsic,  ranging  from  analyses  of  environments,  learners,  content  or  what  have  you. 

What  is  particularly  troubling,  however,  is  that  the  proliferation  of  learning  theories  and  instructional 
methodologies,  despite  all  good  intentions,  have  failed  miserably  at  the  task  of  improving  the  performance 
of  the  designed  instruction.  By  performance,  I don't  necessarily  mean  quantitative  improvements  such  as 
speed  of  learning  or  amount  of  content  and/or  skill,  but  also  qualitative  improvements  such  as  "transfer", 
"retention",  "executive  capacity"  and  the  resulting  promises  of  value  to  our  cultures,  societies  and 
environment  ("quality  of  life"  and  "ethical  standards").  This  should  also  be  extended  to  the  performance  of 
the  instructional  design  process  itself,  of  which  learning  theories  appear  to  be  counter-productive.  For  the 
sake  of  argument,  the  issue  is  perhaps  analyticity  itself,  either  as  over-analysis,  dependency  on  analytical 
frameworks  or  the  mis-use  of  analytical  techniques,  which  is  what  and  how  things  are  measured9. 

Regardless  of  the  instrumental  usefulness  of  analysis,  it  comes  down  to  understanding  where,  when  and 
how  it  is  to  be  deployed.  In  other  words,  an  effective  philosophy  of  education  must  include,  first  and 
foremost,  an  awareness  of  the  consequences  of  analysis,  an  exposure  of  the  underlying  assumptions  behind 
analytical  selection  (which  include  metaphysical  and  ontological  assumptions),  and  ultimately,  epistemic 
guidance  for  the  organization  (interpretation)  of  instruction.  The  idea  driving  this  line  of  thinking  is 
Richard  Rorty's  "post-analytical"  pragmatism10  and  the  project  is  to  apply  this  idea  to  the  common 
analytical  components  of  instructional  design  processes.  While  I borrow  heavily  from  Rorty's  approach  and 
use  the  techniques  of  analytical  philosophers  to  present  my  arguments,  this  is  largely  to  facilitate 
understanding  to  an  academic  audience  and  not  due  to  lack  of  awareness.  The  purpose  is  not  to  deny  or 
radicalize  an  alternative  strategy  (such  as  a "synthetic"  or  "ana-synthetic"  approach)  as  I am  also  in  full 
agreement  with  W.V.O.  Quine's  pragmatism  which  argues  for  a dissolution  of  the  distinction  between  the 
analytic  and  synthetic,  or  to  be  more  precise,  the  division  of  epistemic  statements  into  respective  analytic 
and  synthetic  categories  as  a means  of  determining  truthfulness  (Quine,  1996). 11 


9 The  simple  analogy  is  that  it  is  one  thing  to  have  a theory  to  predict  the  weather  by  means  of  measuring  air 
temperature  and  pressure.  It  is  another  thing  entirely  to  predict  the  weather  by  measuring  the  air  temperature  and 
pressure  in  your  tires. 

10  As  developed  in  Philosophy  and  the  Mirror  of  Nature.  A prescriptive  post-analytical  methodology  is  outlined  later  in 
this  report. 

1 1 The  project  of  neo-pragmatism  is  to  both  look  at  the  consequences  of  traditional  epistemic  dualisms  and  to  dissolve 
what  are  shown  to  be  metaphysical  assumptions.  These  include  mind/body,  fact/value  (or  truth/value),  content/context, 
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The  justification  for  this  approach  is  largely  the  highly  conditional  nature  of  epistemology  as  well  as  human 
activity.  As  demonstrated  by  the  first  part  of  this  article,  any  statement  or  proposition  (basic  or  extended)  is 
effectively  conditional  on  its  possible  contextual  links,  whether  or  not  supplied  by  the  author.  The  truth- 
value  of  any  statement  or  activity  as  evidence  has  been  extensively  discussed  by  (Rorty,  1979).  What  is 
useful  here  is  the  notion  of  a dependency  on  how  our  descriptions  of  things  are  dependent  on  theory12.  The 
distinction  I make  is  not  that  knowledge  is  bound  to  "context"  per  se,  but  that  it  is  conditionally  bound  to 
context  mediated  by  an  interpretive  framework.  The  consequence  of  this  distinction  is  largely  an  issue  of 
how  to  deal  with  value  judgments  upon  the  presentation  of  data. 

The  simple  test  is  to  show  how  analyticity  works,  such  as  in  the  evaluation  of  any  statement  for  truth-value. 
Statement  X is  true  if  it  can  be  verified  through  the  use  of  N reference  forms.  However  the  verification 
argument  falls  down  when  the  dependencies  are  obscured,  and  this  is  what  happens  when  analytical 
approaches  become  the  justification  for  a choice  of  presentation,  or  themselves  used  the  creation  of 
dependencies  for  other  processes.  The  example  would  be  using  a social  constructivist  assumption  about  the 
creation  of  new  knowledge  and  modeling  a learning  environment  after  it.  Just  because  new  knowledge  is 
created  in  a dialogic-social  community  of  practice  is  not  a reason  to  assume  that  this  is  the  best  way  to  learn 
about  pre-existing  knowledge  or  practices.  The  real  issue  is  how  to  prepare  learners  to  be  full-fledged 
participants  in  the  conversations  that  hermeneutically  deal  with  new  data  within  the  boundaries  of  a given 
community. 

Deficiency  Models  as  a consequence  of  "Goal"  Analyses 

Consider  that  most  learning  prescriptions  are  based  on  the  results  of  interpreting  extrinsic  goals  in  a way 
that  presents  a deficiency  that  must  be  corrected.  For  example,  it  is  a purely  metaphysical  assumption  that 
learners  are  containers  that  acquire  knowledge  or  practical  skills.  The  common  "gap"  analysis  establishes 
that  a particular  set  of  knowledge  and/or  skills  are  missing  and  the  instruction  is  designed  to  correct  the 
deficiency.  Entire  domains  of  research  and  productive  enterprises  have  been  built  around  it,  notably 
"performance  improvement",  "designed  instruction".  Other  consequences  of  the  deficiency  analysis  are 
"learning  disabilities"  which  in  turn  spawned  their  own  domains  of  academic  and  economic  activity. 

To  show  how  the  deficiency  model  is  a metaphysically  derived  value-statement  and  not  fact,  consider  that 
"learning  disabilities",  is  just  an  artificial  "categorical"  label,  and  how  it  dissolves  when  we  dispense  with 
the  deficiency  model  of  learning.  A hyperactive  or  inattentive  child  is  considered  "learning  disabled"  as 
their  inattentive  or  hyperactive  nature  somehow  prevents  them  from  acquiring  knowledge  or  skills  at  the 
same  rate  of  success  when  measured  against  their  "normal"  peers.  This  is  a deficiency  analysis,  therefore 
we  find  deficient  results.  The  intuitive  practice  (not  based  on  any  reasonably  accurate  science)  is  to  correct 
the  so-called  "deficiency’'’  that  is  modifying  the  child's  behavior  to  bring  her  up  to  the  performance  level  of 
her  peers.  This  is  clearly  the  consequence  of  a bad  analytical  assumption,  as  it  is  possible  to  turn  this 
around  by  looking  at  other  criteria.  We  could,  as  an  alternative,  choose  to  perceive  the  inattentive  or 
hyperactive  child  as  not  having  the  right  conditions  for  learning13,  or  that  it  is  really  the  environment  or 
learning  strategy  that  is  inappropriate.  For  example,  we  fail  to  explain  how,  and  under  which  conditions, 
learning  "disabled"  students  often  outperform  their  peers  as  they  are  accomplishing  these  feats  under 
"unobservable"  conditions,  usually  outside  externally  directed  learning  contexts. 

If  this  is  indeed  true,  then  we  then  interpret  the  hyperactive  or  inattentive  child  as  actually  a high- 
performance  learner  who  happens  to  be  in  a low-performance  environment.  The  consequence  in  this  case  is 
that  we  have  greater  expectations  on  the  performance  of  our  children,  rather  than  our  learning 


analytic/ synthetic,  appearance/reality  and  so  on.  The  stance  is  skeptical,  in  that  we  are  not  justified  in  accepting  the 
consequences  of  instinctive  dualisms  as  a basis  for  scientific  theories  or  more  precisely,  empiricism. 

12  See  also  Bernard  Lonergan's  notion  of  the  "virtually  unconditioned",  where  he  shows  how  agreed-upon  propositions 
become  virtually  unconditioned  by  "virtue"  of  their  socially  constructed  truth-value.  Once  we  agree  on  a definition  we 
drop  the  conditions  that  make  a proposition  dependent  on  context,  however  those  supporting  assumptions  don't  just 
disappear,  they  are  only  obscured. 

13  This  also  means  questioning  the  validity  of  things  like  Gagne's  first  instructional  event,  which  is  "Gain  Attention". 
Clearly  this  is  a problem  for  nearly  10%  or  more  of  the  student  population  that  can't  even  get  to  the  first  event,  never 
mind  the  other  eight. 
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environments.  We  analyze  the  wrong  criteria  and  call  it  "science".  The  analogy  is  a Ferrari  stuck  in  an 
inner-city  traffic  jam:  this  is  a car  that  will  perform  poorly  in  the  city,  but  on  the  open  road  will  leave  the 
cars  adapted  to  urban  driving  in  its  dust.14  The  point  here  is  do  we  want  to  promote  diversity  of  knowledge 
and  skill  sets,  or  homogenize  the  learners  to  better  fit  the  system? 

The  obfuscation  of  extrinsic  goals  in  the  instructional  design  process 

Extrinsic  goals  are  defined  here  as  an  integral  part  of  the  needs  analysis  process,  whereby  the  reasons  or 
needs  for  a learning  package  are  defined.  These  are  referred  to  also  as  business  goals  or  stakeholder  goals, 
or  the  motivation  behind  the  designed  instruction.  These  are  not  to  be  confused  with  learning  objectives  or 
goals  (which  have  more  to  do  with  internal  measurement  or  as  the  plan  to  get  through  the  learning  content). 

An  analytical  goal  inquiry  usually  aligns  a content  or  skill-based  competency  with  a "business"  need, 
whether  it  is  poor  performance  or  relative  performance,  strategic  needs  such  as  equity  building,  resource 
exploitation  or  competitive  pressures,  or  compliance  needs  such  as  legal  or  mandated  safety  standards. 

What  is  not  obvious  is  how  deeply  layered  and  multi-faceted  extrinsic  goals  compete  to  shape  the 
motivation  behind  a learning  package.  For  example,  the  federal  government  might  have  a goal  of 
improving  Canada's  ranking  in  high-school  science  and  math  education,  which  is  transferred  and 
interpreted  by  the  provincial  governments  responsible  for  curriculum,  which  is  then  transferred  and 
interpreted  to  regional  educational  school  boards,  then  on  to  the  schools,  then  the  teachers  and  finally  the 
students.  The  next  line  is  society  goals,  which  might  include  the  needs  of  various  cultural  groups  or  other 
structures  such  as  the  employment  market,  and  finally  the  needs  and  expectations  of  parents.  It  is  not  likely 
in  any  stage  that  the  root  extrinsic  goal  is  valid  given  the  environmental  (or  contextual)  constraints  facing 
each  stakeholder  having  an  influence  on  actual  learning  outcomes,  right  down  the  line.  It  is  also  a huge 
stretch  of  the  imagination  that  this  goal  is  attainable  not  so  much  because  of  these  constraints  but  rather  that 
each  step  introduces  process  goals  that  supervene  or  take  priority. 

This  is  not  to  say  that  all  of  these  goals  are  competing.  For  example,  it  is  easy  for  stakeholders  to  accept 
literacy  goals  as  virtually  all  stakeholders  value  highly  the  benefits  that  literacy  brings.  But  even  with 
adequate  goal  alignment,  literacy  levels  in  Canada  have  actually  declined  from  1975  to  1990  and  only 
showing  some  improvement  since  then  (Coulombe,  Tremblay,  & Marchand,  2006).  This  is  an  example  of 
how  constraints,  or  rather  the  competitive  goals  of  the  sub  processes  of  the  system  conspire  to  obscure  the 
laudable  goal  of  literacy.  These  are  usually  complaints  of  "lack  of  money,  time  or  human  resources"  which 
just  a way  of  saying  that  provincial  governments,  school  boards,  schools,  teachers  and  students  all  have 
other  priorities,  thus  other  goals,  for  the  resources  at  hand. 

A post- analytical  goal  inquiry  seeks  to  answer  why?  questions  with  the  following  dimensions:  Why  is  it  we 
organize  grade  9 algebra  to  begin  and  end  at  a given  point?  Why  do  we  wait  until  grade  9 to  introduce 
algebraic  concepts?  Why  do  we  feel  that  all  students  must  master  algebraic  operations?  In  other  words,  the 
same  as  "Miss,  WHY  do  we  have  to  learn  this  stuff. ? " as  any  high  school  teacher  might  be  typically  asked  by 
her  students  when  giving  them  all  of  this  abstract  content.  Poor  "miss"  can  give  all  kinds  of  plausible 
explanations  about  developmental  psychology,  requisite  knowledge  or  skills,  or  that  it  is  needed  to  get  a job 
or  to  graduate  from  high  school,  but  the  reality  is  that  the  true  answers  are  formed  by  the  needs  of  the 
educational  structure  (Bruner,  1996)  and  then  obscured  by  the  immediate  need  of  getting  classroom 
students  to  the  end  of  the  course  content  so  they  can  pass  the  exam.  We  don't  dare  admit  that  immediate, 


14  This  is  not  to  say  that  we  do  not  have  a humane  way  of  dealing  with  the  issue  of  attention-deficit/hyperactivity 
disorder.  Just  as  most  Ferraris  don't  ever  see  a racetrack,  we  prevent  their  high-performance  engines  from  stalling  from 
the  effect  of  extended  idling  (fouled  spark  plugs)  by  simply  raising  the  idle  speed.  This  is  exactly  what  stimulant 
medication  does  with  the  ADHD  child,  it  effectively  raises  the  "idle  speed"  of  their  brains  and  prevents  their  neuronal 
"spark  plugs"  from  fouling.  What  is  not  explicitly  stated  is  that  most  children  have  an  adaptive  capacity  for  extended 
"idling"  in  the  typical  classroom  but  it  does  not  justify  the  use  of  learning  environments  that  are  low-performance  as 
consequential  and  too  costly  to  change.  As  with  learners,  cars  simply  perform  better  when  not  slowed  down  by  traffic 
congestion  and  other  barriers  to  movement.  However  it  just  appears  easier  to  change  the  student  and  not  the  system. 
The  consequence  is  billions  of  dollars  worth  of  pharmaceuticals  are  purchased  by  parents  and  billions  of  barrels  of  oil 
are  wasted  through  unnecessary  traffic  congestion.  If  schools  were  made  responsible  for  the  cost  of  all  the  stimulant 
drugs  prescribed,  things  would  surely  be  different.  The  reality  is  that  we  do  not  hold  schools  accountable  for  poor 
performance. 
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practical  needs  take  precedence  over  the  meta-goal  so  we  come  up  with  excuses  in  the  form  of  "plausible 
explanations"  that  have  no  grounding  in  anything  we  could  even  reasonable  call  "science".  The 
consequence  is  that  the  content  becomes  ambiguous  as  there  is  little  solid  pragmatic  context  in  which  to 
relate  it  to,  other  than  it  is  needed  to  get  to  Calculus  or  Engineering  or  it  is  good  for  cognitive  development. 
The  obfuscation  of  goals  is  also  apparent  in  the  type  of  instructional  design  seen  in  business  training. 
However  it  would  seem  that  it  should  be  so  much  easier  as  there  are  fewer  stakeholders  and  goals  are  fewer 
and  easier  to  identify  (profit,  customer  satisfaction,  lower  turnover,  etc.).  However  business  training  is  just 
as  ineffective  as  process  goals  are  still  interfering  and  taking  precedence.  For  example,  a company  might 
want  to  increase  sales  performance,  however  the  best  way  to  train  salespeople  might  not  be  possible  due  to 
the  high  cost  of  development  of  training  (a  competing  goal  of  keeping  expenses  down)  or  the  technological 
infrastructure  (a  competing  goal  of  the  needs  of  the  IT  department  to  have  a reliable,  purpose-built  system). 
Even  if  the  constraints  are  overcome,  there  is  no  consensus  on  how  to  interpret  "increased  sales"  into 
employee  performance  behavior,  and  competitive  pressures  or  production  inefficiencies  might  be  casting  a 
big  shadow  on  an  analysis  that  is  just  too  narrow  to  consider  it  a factor  in  increased  sales  per  employee. 

The  obfuscation  of  extrinsic  goals  has  a direct  relationship  also  to  the  selection  and  presentation  of 
instructional  content,  which  usually  includes  other  analyses  as  the  subject  matter  expert  (SME),  the  learner, 
the  environment  in  which  the  learning  takes  place  and/or  eventually  used.  Each  of  these  factors  will  be 
interpreted  with  some  kind  of  framework  and  learning  content  then  selected  and  evaluated  to  meet  the 
relevant  criteria  for  a given  learning  strategy.  Each  analytical  result  is  the  product  of  the  goal  of  the 
interpretive  framework,  and  therefore  takes  precedence  over  the  extrinsic  goals  in  the  selection  of  content. 
Let's  not  forget  the  poor  learner  who  is  now  faced  with  trying  to  come  to  terms  with  the  content  or  activity 
that  has  been  selected  for  him.  Most  of  it  is  irrelevant  to  his  goals,  we  then  spend  time  and  resources  trying 
to  get  his  goals  aligned  with  the  business  goals  (incentives  and  consequences)  which  are  by  now 
completely  subsumed  by  the  practical  concerns  of  delivering  the  instruction. 

Finally  it  is  worth  mentioning  that  evaluation  is  also  a constraint  that  competes  for  content/activity 
selection,  in  the  sense  that  we  strive  for  fair  and  objective  assessments  of  learners.  Objective  assessments 
must  be  easy  to  measure,  therefore  observable  and  interpretable  in  a way  that  is  reliable.  Even  though  a 
particular  activity  or  content  might  be  the  most  appropriate  for  a given  competency  goal,  it  may  not  be 
selected  just  due  to  the  interference  of  assessment  goals. 

The  intuition  is  that  excessive  time  spent  in  interpretive  activities  is  interfering  with  the  learning  process, 
therefore  an  anti-interpretive  instructional  design  methodology  might  improve  learning  performance. 
The  way  out,  as  1 have  suggested,  might  be  to  adopt  an  anti-interpretive  approach  to  designing  instruction. 
The  intent  is  to  remove  interference  between  the  learner  and  the  desired  knowledge/competency  regardless 
of  the  motivation  to  learn  or  the  directives  of  getting  other  people  to  learn. 

This  could  be  as  simple  as  making  all  of  the  extrinsic  goals  explicit,  and  guide  learners  in  developing  their 
own  personalized  interpretations  of  those  goals.  For  example,  it  makes  better  sense  to  do  this  rather  than 
provide  hit-or-miss  interpretations,  the  results  being  that  more  time  is  spent  interpreting  interpretations  and 
dealing  with  the  consequences  of  doing  just  that.  The  consequence  is  spending  more  time  managing  the 
process  of  interpretation  of  pre-inteipreted  content,  rather  than  learning  how  to  apply  the  knowledge.  The 
hit-or-miss  "predigested"  instructional  content  approach  misses  the  point  that  learning  about  is  answering 
why  questions  with  a much  broader  scope,  or  the  type  of  scope  that  is  commonly  employed  by  the 
analytical  approach  of  instructional  design.  To  move  beyond  content  acquisition,  which  now  becomes 
appropriated  beliefs,  as  guided  by  the  answering  of  personally  relevant  why  questions,  is  a strong  intrinsic 
motivator  for  human  activity  (Carley,  2005)  (Lonergan,  1970),  and  according  to  social  constructivism,  it  is 
through  activity  that  knowledge  is  created  and  where  learning  is  most  effective. 

Rorty's  Post-analytics  as  a model  for  the  anti-interpretation  approach 

The  approach  I am  advocating  is  pragmatic,  or  to  be  more  specific,  neo-pragmatic  in  that  the  key  is 
uncovering  the  underlying  assumption  of  social  practice  without  regard  for  dualistic  interpretations.  In  this 
way  neo-pragmatism  is  at  once  holistic  without  denying  the  productive  qualities  of  descriptive  frameworks 
that  might  be  considered  dualistic  in  nature.  The  other  characteristic  of  neo-pragmatism  is  that 
consequential  results  are  sought  and  aligned  with  identified  value  propositions,  rather  than  truth- 
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verification.  Finally  it  is  understood  that  the  important  thing  about  conceptual  knowledge  is  not  cultural 
relativism  (where  truth  statements  are  verified)  but  that  simply  concepts  are  unstable.  These  points  of  view 
have  been  developed  by  Quine,  Davidson,  and  Rorty  and  informed  by  the  analytical  methods  of 
Wittengstein,  Habermas  and  Gadamer.  Post-analytics  could  be  considered  the  methodologies  of  both 
Derrida  (deconstruction)  and  Foucault  (genealogy)  which  have  admittedly  different  aims  but  are 
nonetheless  useful.  Richard  Rorty's  post-analytical  pragmatism  (Rorty,  1979)  (Rorty,  1982)  as  it  touches 
learning  and  epistemology,  can  be  defined  as  follows: 

1.  When  encountering  anything  that  is  outside  our  current  knowledge  structures  (i.e.  rationalist  or 
empirically  derived)  (new  data,  new  phenomena)  we  automatically  assume  a "hermeneutical"  stance  in 
order  to  deal  with  it. 

2.  All  analyses  are  derivative  and  reductive,  played  out  as  a language  game. 

3.  The  Cartesian  mind-body  dualism  is  not  sufficient  for  the  development  of  a solid  epistemology. 

4.  Apprehension  (coming  to  know)  is  imperfect,  therefore  concepts  are  not  stable. 

5.  Justification  for  beliefs  is  a matter  of  social  (and/or  cultural)  practice. 

As  this  pertains  to  an  anti-interpretive  approach,  this  means  several  things. 

1 . An  understanding  that  content  analysis  serves  little  purpose  but  to  create  ambiguity,  compounded  by  the 
competing  goals  of  the  entire  instructional  process,  including  extrinsic  goals.  The  approach  is  not  to 
dispense  with  content  but  to  dissolve  the  distinction  between  knowledge  and  skill,  or  knowledge  and  its 
application.  It  makes  little  sense  to  guess  "requisite  knowledge"  for  a certain  skill  set  and  present 
content  before  it  can  be  used.  This  is  perhaps  a mirror  of  how  it  works  in  social  practice  but  not 
necessarily  in  a learning  context.  Instructional  design  needs  to  work  with  the  hermetical  stance  of  the 
learner  and  not  against  it. 

2.  The  needs  of  the  systems  of  education  should  not  supervene  on  the  learning  need.  We  could  call  this 
"goal  alignment"  but  it  is  really  about  aligning  the  goals  of  the  system  (and  not  the  learners).  An 
assessment  of  any  ID  process  should  be  able  to  expose  how  interpretation  of  system  needs  undermines 
learning. 

3.  The  implications  of  giving  up  the  mind/body  dualism  are  huge  and  not  really  useful  for  the  design  of 
instruction  other  than  to  understand  that  it  does  not  produce  a solid  epistemology.  This  is  meant  to  be 
shown  by  the  instability  and  abstract  nature  of  conceptual  knowledge  as  somehow  "discrete"  objects 
that  are  stored  as  representations  in  a place  called  the  "mind".  Rorty's  response  is  epistemological 
behaviorism,  which  is  to  say  we  should  rather  study  the  behavior  of  knowledge/activity  and  how  it 
comes  about  rather  than  working  out  better  and  better  representations  of  it,  or  by  analyzing  and 
attempting  to  modify  individual  behavior  as  a teaching  strategy. 

4.  Since  apprehension  is  not  perfect,  it  makes  little  sense  to  use  it  as  an  implicit  learning  goal.  This  has  the 
added  bonus  of  dispensing  with  learning  assessments  that  are  designed  to  measure  apprehension,  and 
the  resulting  interpretive  effect  on  content/activity  selection. 

5.  Justification  for  belief  replaces  factual  statements  that  are  meant  to  be  verified  by  direct  correspondence 
to  experience  or  to  nature.  By  the  same  token  this  proposition  does  not  obscure  value  propositions 
behind  statements  of  "objective"  fact,  or  what  might  have  been  derived  empirically  or  through  other 
forms  of  analysis.  A post  analytical  needs  assessment  considers  primarily  what  is  valued  as  the  meta- 
goal driving  the  need  for  instruction,  and  explores  the  consequences  of  those  values  when  put  through 
instructional  processes. 

Where  to  from  here 

This  methodology  is  nascent  and  admittedly  not  fully  worked  out.  There  are  many  other  consequences  to 
this  approach,  including  how  affective  approaches  to  learning  might  be  incorporated  with  this 
methodology.  For  example,  if  we  dispense  with  the  positivistic-analytical  approaches  to  learning  we  can 
reintroduce  a huge  component  of  human  experience,  which  is  how  we  feel  and  this  does  not  preclude 
learning  about  them.  In  addition,  the  process  of  learning  is  inherently  pleasurable  and  this  begins  with 
curiosity,  or  the  desire  to  know  and  the  intense  pleasure  at  experiencing  Eureka!  moments.  Our  excuse  has 
been  it  is  difficult  to  observe  and  measure  affective  outcomes,  but  if  does  not  mean  we  cannot  use  how  we 
feel  as  a gateway  to  the  use  of  interpretive  frameworks  to  rationalize  data.  The  goal  is  not  an  emotional 
outcome  as  a learning  goal  (that  is  silly  and  why  we  don't  do  it)  but  to  use  emotional  response  as  an 
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instructional  strategy.  If  the  mind/body  distinction  is  obliterated,  then  so  should  the  emotional/rational. 
Since  we  are  linking  knowledge  to  belief  in  this  strategy  we  also  acknowledge  that  the  strength  of  belief  is 
something  that  is  highly  charged  with  emotion. 


The  other  intuition  is  that  real,  qualitative  and  quantitative  research  can  be  done.  I believe  that  tools  could 
be  developed  that  can  help  assess  existing  instruction  for  the  amount  of  interpretive  load  placed  on  a 
learner.  The  development  and  refinement  of  a post-analytical  "anti-interpretation"  instructional  design 
strategy  would  also  benefit  from  research.  It  may  seem  odd  that  I appear  to  be  advocating  the  dropping  of 
analytical  techniques  but  at  the  same  time  talking  about  creation  of  an  analytical  tool  to  measure  the  effects 
of  analysis.  However  this  is  not  really  the  case,  as  this  is  about  the  interpretive  effect  of  analyticity  and  the 
solution  is  how  to  balance  it,  not  to  radicalize  the  alternative.  As  much  as  anti-interpretation  might  sound 
radical,  it  is  really  about  giving  equal  time  to  the  range  of  human  experience  that  has  been  shoved  aside  in 
the  name  of  positivism,  the  rejection  of  metaphysics  in  the  psychological  approaches  to  education  and  the 
analytical  approaches  to  designing  and  presenting  instruction. 
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Abstract 

The  purpose  of  this  study  is  to  examine  the  extent  on  how  middle  school  teachers  value  the  technology  in  classroom 
and  to  what  degree  their  students  are  using  technology  and  the  facilities,  equipment  provided  by  their  schools.  To 
answer  many  questions  surrounding  technology  in  schools,  a study  was  conducted  with  randomly  selected  one 
thousand  middle  schools  in  eleven  states  and  the  data  was  conducted  at  the  end  of  April  2004. 

Introduction 

Few  would  disagree  that  technology  is  increasingly  important  in  the  lives  of  all  Americans.  In  education, 
especially,  the  use  of  technology  has  grown  tremendously.  For  example,  the  number  of  computers  in  schools 
continues  to  rise  each  year  with  more  than  fourteen  million  computers  in  our  elementary  and  secondary  schools 
(U.S.  National  Center  for  Education  Statistics,  2003-2004).  The  technology  skills  of  students  have  also  grown 
dramatically.  A major  national  report  on  technology  and  readiness  in  schools,  emphasized  that  all  students  must 
graduate  from  high  school  with  the  technology  skills  needed  for  life  in  the  21st  century.  All  teachers  must  also  be 
able  to  use  technology  as  a major  tool  for  achieving  higher  academic  standards  in  their  classrooms  (CEO  Forum, 
1999).  In  response  to  the  call  for  more  and  more  technology  in  schools,  most  states  have  adopted  technology 
standards  for  those  preparing  to  be  teachers.  The  National  Council  for  Accreditation  of  Teacher  Education 
(NCATE,)  in  its  2000  Unit  Standards,  advised  that  all  programs  for  educating  new  teachers  must  produce  candidates 
who  are  able  to  use  educational  technology  to  help  all  students  learn.  Programs  must  show  how  knowledge,  skills, 
and  dispositions  related  to  educational  and  information  technology  are  integrated  throughout  the  curriculum, 
instruction,  field  experiences,  clinical  practices,  assessment,  and  evaluation  (NCATE,  2001  p.  13).  Most  experts 
would  agree  that  all  schools  should  help  students  be  technologically  skilled  to  face  the  challenges  of  the  new 
century. 

In  spite  of  the  tremendous  growth  of  technology  in  American  schools  there  is  evidence  that  many  teachers  still 
do  not  use  technology  at  all  in  their  teaching  (Education  Week,  1999).  Many  questions  also  exist  regarding  the  real 
values  of  technology  in  schools.  For  example,  are  schools  providing  enough  tools,  hardware  and  software  for 
students  to  learn  well?  Do  teachers  really  value  technology  tools  in  their  teaching?  In  spite  of  these  concerns,  most 
modern  educators  believe  that  technology  should  play  a major  role  in  today's  schools.  Middle  schools,  in  particular, 
can  help  their  students  develop  the  technology  skills  needed  for  success  in  later  life  and  at  the  same  time  increase 
their  achievement. 

To  answer  the  many  questions  surrounding  technology  use  in  middle  schools,  a survey  was  sent  to  1000  middle 
schools  in  ten  states  (New  Hampshire,  Vermont,  West  Virginia,  Kentucky,  Oklahoma,  Texas,  South  Dakota, 
Massachusetts,  Montana,  Nevada,  and  Iowa).  The  study  explored  the  types  of  technology  used  by  middle  school 
teachers,  how  they  valued  the  technology,  and  the  facilities  and  equipment  provided  by  their  schools.  One  thousand 
questionnaires  were  sent  to  these  teachers  and  three  hundred  sixty-nine  were  returned. 

The  questionnaire  consisted  of  two  parts.  In  part  one  the  teachers  marked  how  often  they  used  particular  types 
of  technology  (never,  rarely,  sometimes  and  often)  and  how  valuable  these  types  of  technology  were  in  their 
teaching  (very  valuable,  valuable,  no  opinion,  little  value  and  no  value).  In  part  two  the  teachers  indicated  the 
availability  of  different  types  of  technology  in  their  schools  and  the  training  provided  for  them  and  their  students  by 
their  schools.  The  results  were  analyzed  using  percentages  of  use  and  of  value.  The  majority  of  those  responding 
were  female  with  one  to  ten  years  of  experience.  Moreover,  most  of  the  respondents  were  mathematics  and  science 
teachers  followed  by  those  teaching  social  studies  and  English.  This  article  will  report  only  those  findings  dealing 
with  the  values  that  teachers  had  toward  various  types  of  technology  in  their  classrooms. 
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Results 


Table  A.  Values  of  Technologies  Used  by  Teachers 


Types  of  Technology 

Ratings 

Percent  of  Teachers 

Word  Processing 

Very  Valuable 

74.5% 

Computers  for  preparing  lessons 

Very  Valuable 

69.9% 

Internet 

Very  Valuable 

60.4% 

VCR 

Valuable 

55.6% 

CD-ROM 

Valuable 

55.6% 

Television 

Valuable 

46.1% 

Digital  Cameras 

Valuable 

40.9% 

Computer  graphics 

Valuable 

40.4% 

E-Mail 

Valuable 

62.9% 

Web  pages 

Valuable 

35.5% 

Films 

Valuable 

41.7% 

Digital  Camcorders 

Valuable 

39.0% 

Microsoft  PowerPoint 

Very  Valuable 

48.0% 

Tape  Recorders 

Valuable 

34.4% 

Laser  Disk  Players 

Valuable 

25.5% 

Radios 

Valuable 

20.1% 

Slide  Projectors 

Valuable 

21.4% 

Filmstrips 

Valuable 

16.0% 

Table  A shows  that  the  types  of  technology  most  valued  by  teachers  were:  word  processing  (74.5%  very  valuable), 
computers  for  preparing  lessons  (69.9%  very  valuable),  and  the  internet  (60.4%  very  valuable).  The  next  most 
valued  types  of  technology  with  ratings  of  valuable  were:  VCR's  (55.6%),  CD  ROM's  (55.6%),  television  (46.1), 
digital  cameras  (40.9%),  use  of  graphics  for  teaching  (40.4%),  e-mail  for  discussing  teaching  with  other  educators 
(62.9%),  web  pages  (35.5%),  fdms  (41.7%),  digital  camcorders  (39.0%),  and  Microsoft  Power  Point  (48%).  Other 
less  valued  technological  aids  which  received  valuable  ratings  were:  tape  recorders  (34.4%),  laser  disc  players 
(25.5%),  radios  (20.1%),  slide  projectors  (21.4%),  and  filmstrips  (16.0%). 


Table  B.  Values  of  Technologies  Required  or  Encouraged  for  Students 


Types  of  Technology 

Ratings 

Percent  of  Teachers 

Encourage  the  use  of  the  Internet  by  students 

Very  Valuable 

61.4% 

Require  the  use  of  computers  for  learning  or 
assignments 

Very  Valuable 

57.1% 

Require  the  use  of  the  Internet  by  students 

Very  Valuable 

50.8% 

Encourage  the  use  of  digital  cameras  by  students 

Valuable 

42.9% 

Require  the  use  of  word  processing  by  students  for 
completing  assignments 

Valuable 

45.7% 

Encourage  the  use  of  computer  graphics  by  students 

Valuable 

44.2% 

Encourage  the  use  of  CD  ROM's  by  students 

Valuable 

43.9% 

Encourage  the  use  of  E-Mail  by  students 

Valuable 

35.7% 

Encourage  the  use  of  spreadsheets  by  students 

Valuable 

42.1% 

Encourage  the  use  of  databases  by  students 

Valuable 

31.5% 

Encourage  the  use  of  electronic  portfolios  by  students 

Valuable 

33.0% 

Table  B reveals  how  teachers  felt  about  the  use  of  technology  by  their  students.  The  most  valued  types  of 
technology  required  by  teachers  were:  using  computers  for  learning  or  assignments  (57.1%)  and  using  the  Internet 
(50.8%%  very  valuable).  In  addition,  those  tools  encouraged  by  teachers  were:  the  use  of  the  Internet  (61 .4%  very 
valuable),  digital  cameras  (42.9%  valuable),  word  processing  for  completing  assignments  (45.7%  very  valuable), 
computer  graphics  (44.2%  valuable),  CD-ROM's  (43.9%  valuable),  e-mail  use  (35.7%  valuable),  spreadsheets 
(42.1%  valuable),  database  (31.5%  valuable),  and  electronic  portfolios  (33.0%  valuable). 
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Conclusions 


The  results  of  this  study  clearly  showed  the  value  of  certain  technology  tools  used  in  middle  schools.  It  was 
clear  that  only  three  forms  of  technology  (computers,  the  Internet  and  word  processing)  were  seen  as  very  valuable 
by  most  (over  60%)  of  the  teachers.  Other  technological  aids  (web  pages,  digital  cameras,  VCRs  and  films)  were 
rated  as  valuable  by  less  than  40%  of  the  teachers  and  e-mail  was  rated  valuable  by  63%.  Other  types  of  technology 
(Microsoft  Power  Point  presentations,  tape  recorder,  laser  disk  players,  radios,  slide  projectors  and  film  strips)  was 
rated  as  valuable  by  even  a lower  numbers  of  teachers  (16%  to  48%).  This  apparent  lack  of  strong  support  for 
technological  aids  in  middle  schools  is  somewhat  surprising  in  light  of  the  many  proven  benefits  of  these  aids  for 
teachers  and  students. 

Additional  findings  affirm  that  middle  school  teachers  tend  to  encourage,  not  require,  the  use  of  technology  by 
their  students.  As  was  the  case  with  the  teachers,  only  computers  and  the  Internet  were  rated  as  very  valuable  by 
51%  to  61%  of  the  teachers.  Other  technological  aids  (word  processing,  computer  graphics,  CD-ROMs,  e-mail, 
spreadsheets,  databases  and  electronic  portfolio)  were  seen  as  valuable  by  fewer  teachers  (32  to  46%).  Once  again, 
these  results  showed  limited  support  for  student  use  of  technology. 

Findings  from  this  study  also  showed  that  middle  school  teachers  did  value  specific  technological  aids  in  their 
teaching,  but  these  aids  were  not  overwhelmingly  supported  by  most  of  the  teachers.  It  is  further  apparent  that  many 
middle  school  teachers  did  encourage  their  students  to  use  technology  for  their  learning.  The  number  of  teachers 
who  did  encourage  their  students  to  use  technology  ranged  greatly,  from  32%  to  46%,  depending  upon  the  particular 
type  of  technology.  It  was  not  surprising  that  computers  and  the  Internet  were  the  most  popular  types  of  technology 
for  both  teachers  and  students,  since  many  teachers  consider  the  term  "technology"  to  really  mean  computers  and 
their  use  in  the  classroom.  It  will  undoubtedly  take  time  for  other  types  of  technology  to  increase  in  popularity 
among  teachers.  As  greater  numbers  of  middle  schools  provide  more  technological  aids  and  opportunities  for  in- 
service  training,  the  attitudes  of  middle  school  teachers  will  hopefully  become  more  positive  toward  all  types  of 
technology  in  their  classes. 
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Abstract 

Mentoring  is  an  important  aspect  of  faculty  life.  Most  universities  have  a single  campus,  so  mentoring  is 
easily  facilitated.  What  happens  when  the  faculty  is  spread  across  a large  state?  This  session  reports  on  a faculty 
mentoring  program  that  attempts  to  close  the  distance  gap  by  organizing  Faculty  Learning  Communities  and  using 
distance  learning  technology  (iLinc).  This  session  is  beneficial  to  those  who  are  mentoring  or  thinking  about 
mentoring  faculty. 

Mentoring 

Theoretical  support  can  be  found  for  establishing  mentoring  programs  for  university  faculty.  Effective 
mentoring  promotes  professional  and  collegial  faculty  who  provide  consistent  and  rigorous  academic  programs  for 
students  within  the  university.  In  addition,  mentoring  programs  at  colleges  and  universities  can  be  a model  to  ensure 
successful  transition  of  new  faculty  into  their  academic  environment.  These  programs  are  usually  based  on  the 
traditional  sort  where  a seasoned  faculty  facilitates  the  development  of  persons  identify  as  proteges.  A study  of 
recent  literature  reveals  there  is  limited  information  on  co-mentoring  or  faculty-to-faculty  mentoring.  Co-mentoring 
models  that  are  operational  in  higher  education  demonstrate  professionally  equal  relationships  and  expand  upon 
those  that  are  exclusive  (Mullen  & Lang,  2005).  The  authors  of  this  study,  have  launched  a learning  community  with 
the  specific  goal  of  mentoring  faculty  who  have  expressed  a desire  to  take  an  active  role  in  their  professional 
growth. 

A faculty  mentoring  program  is  useful  in  providing  structure  for  new  faculty  as  they  begin  their  careers  at 
the  university  level.  Mentoring  is  viewed  as  a strategy  for  enhancing  the  development  and  refinement  of  one’s 
ability  to  transition  into  a new  academic  culture  and  for  the  university  to  ensure  that  students  benefit  through  highly 
effective  faculty  (Wunsch,  1994).  The  mentoring  process  helps  to  build  a collegial  and  cooperative  relationship, 
which  fosters  a more  productive  institutional  environment  Boyle  & Boice,  1998). 

Universities  benefit  through  increased  productivity  of  their  faculty  to  ensure  the  success  of  their  students 
(Nicholls,  2002).  Various  colleges  and  universities  have  established  mentoring  programs  for  new  faculty  members: 
University  of  California,  San  Diego,  Southern  Illinois  University,  and  University  of  Hawaii,  to  name  a few 
(Wunsch,  1994).  Some  of  these  programs  have  clearly  defined  roles  and  responsibilities  with  long  and  short-term 
goals.  Successful  mentoring  programs  vary  in  type.  They  can  be  formal  or  informal,  organized  at  the  department, 
school  or  institutional  level. 

Curiously,  there  is  a parallel  in  the  principles  of  mentoring  with  learning  community  programs.  The 
fundamental  purpose  of  learning  communities  is  to  increase  coherence  in  and  between  subject  matters,  to  reverse 
intellectual  fragmentation,  which  many  disciplines  tend  to  promote,  and  to  encourage  interaction  between  faculty 
and  students,  among  students  themselves,  and  among  faculty  team  members  (Regional  Learning  Communities 
Consortium,  2004).  Faculty  learning  communities  are  different  from  but  in  many  ways  like  most  action  learning  sets 
in  that  they  both  are  “a  continuous  process  of  learning  and  reflection,  supported  by  colleagues,  with  an  intention  of 
getting  things  done”  (McGill  & Beaty,  2001,  p.  1 1). 

Faculty  Learning  Communities  (FLCs)  are  cross-disciplinary  learning  communities.  There  are  two 
categories  of  FLCs:  cohort-based  and  topic-based.  Cohort-based  FLCs  “address  the  teaching,  learning,  and 
developmental  needs  of  an  important  grop  of  faculty  or  staff  that  has  been  particularly  affected  by  the  isolation, 
fragmentation,  stress,  neglect,  or  chilly  climate  in  the  academy”  (Cox,  2004,  p.8).  .A  curriculum  for  this  type  of 
learner  cohort  would  focus  on  teaching  and  learning  topics  that  interest  them.  Topic-based  FLCs  “have  a curriculum 
designed  to  address  a special  campus  teaching  and  learning  need,  issue,  or  opportunity  (Cox,  2004,  p.8). 

Participants  in  the  survey,  conducted  in  the  fall  of  2003,  were  full-time  faculty  members  at  a private 
university  in  California  was.  From  that  survey,  the  need  for  mentoring  was  ranked  at  the  top  of  issues  to  be 
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addressed.  After  a review  of  current  research,  a follow  up  presentation  on  mentoring  was  given  at  the  Faculty  Spring 
Symposium  in  April,  2004.  Input  was  received  from  20  participants  in  the  session  as  to  what  their  “Vision  of 
Mentoring”  would  be  at  the  University. 

Ideas  included  (stated  as  written  by  participants):  training  for  the  mentor,  natural  selection  for  both  mentor 
and  mentee,  recognition  by  the  university  as  a Faculty  Service  component,  mentoring  should  be  done  within 
departments,  should  have  regularly  scheduled  times  for  meeting  both  formally  and  informally,  need  to  set  up  a 
university  model,  administrative  support  necessary  and  guidelines  set  forward  in  a Mentor  Handbook,  set  up  a 
minimum  of  times  to  meet,  and  identify  people  in  departments  who  are  experts  to  serve  as  mentors. 

At  the  Faculty  Spring  Symposium  in  May  2005,  a survey  was  administered  to  faculty  to  further  define  the 
needs  for  mentoring  of  professors  at  various  stages  in  their  careers.  The  response  from  107  faculty  or  55  % of  the 
total  faculty  indicated  their  initial  areas  of  interest  were  writing  opportunities,  specifically  the  publishing  process, 
and  preview  and  feedback  on  materials  for  publishing.  The  second  area  was  the  desire  to  increase  technology-related 
skills,  particularly  technology  in  the  university  classroom  and  online  teaching  techniques. 

Our  university  has  established  a Center  for  the  Adult  Learner  (CAL).  The  goal  of  this  center  is  to  foster 
best  practices  in  the  instruction  of  adults.  One  of  the  first  efforts  sponsored  by  CAL  is  Faculty  Learning 
Communities  (FLCs).  These  FLCs  promote  scholarship  among  members  and  sharing  successful  strategies  for 
enhancing  teaching  and  leaning  in  all  courses.  Based  on  the  data  gathered,  a new  faculty  learning  community  was 
launched  in  Fall  2005  with  the  specific  goal  of  mentoring  faculty  in  using  technology  in  the  university  classroom. 
Because  the  faculty  is  spread  across  27  sites  a distributed  learning  strategy  was  determined  to  be  the  most  efficient 
way  to  conduct  group  meetings.  Group  members  use  iLinc  software  to  meet  each  month.  With  this  conferencing 
software,  meeting  attendees  can  see  and  hear  each  other. 

In  its  first  year  the  FLC  had  5 members.  The  interests  of  the  groups  were  divided  into  two  areas:  using 
different  software  and  changes  to  the  andragogy  because  of  the  integration  of  technology.  Members  were  able  to 
share  techniques  for  using  different  software.  There  were  also  meaningful  conversations  about  increasing  and 
enhancing  interaction  in  online  classes. 
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What  is  a Podcast?  / What  is  Podcasting? 


A Google  search  > define:  Podcast 
Results  in: 

Podcasting,  a portmanteau  of  Apple's  "iPod"  and  "broadcasting",  is  a method  of 
publishing  files  to  the  Internet,  allowing  users  to  subscribe  to  a feed  and  receive  new  files 
automatically  by  subscription,  usually  at  no  cost.  It  first  became  popular  in  late  2004, 
used  largely  for  audio  files. 

Wikipedia  defines  podcating  as: 

Podcasting  is  the  distribution  of  audio  or  video  files,  such  as  radio  programs  or  music 
videos,  over  the  Internet  using  either  RSS  or  Atom  syndication  for  listening  on  mobile 
devices  and  personal  computers.  A podcast  is  a web  feed  of  audio  or  video  files  placed 
on  the  Internet  for  anyone  to  download  or  subscribe.  Podcasters'  websites  also  may  offer 
direct  download  of  their  files,  but  the  subscription  feed  of  automatically  delivered  new 
content  is  what  distinguishes  a podcast  from  a simple  download  or  real-time  streaming 
(see  below).  Usually,  the  podcast  features  one  type  of  "show"  with  new  episodes  either 
sporadically  or  at  planned  intervals  such  as  daily,  weekly,  etc.  Besides  that  there  are 
podcast  networks  that  feature  multiple  shows  on  the  same  feed.  Podcasting's  essence  is 
about  creating  content  (audio  and/or  video)  for  an  audience  that  wants  to  listen  when  they 
want,  where  they  want,  and  how  they  want. 

Podcasting  In  General 

Creating  a podcast  requires  very  careful  planning.  The  better  or  tighter  your  planning 
phase  is,  the  smoother  the  actual  production  of  the  podcast  will  go.  Technology  needed 
to  develop  a podcast  are:  1)  A desktop  or  laptop;  2)  A microphone  or  digital  media 
recorder  such  as  an  iRiver  or  a MiniDisc  Recorder;  3)  Audio  Editing  Software,  such  as 
Audacity  or  GarageBand;  and  4)  An  application  to  help  you  create  the  “enclosures”  of  the 
podcast,  such  as  Podifier.  There  are  many  other  alternatives  to  the  technology  mentioned 
above,  but  for  the  sake  of  this  paper,  and  our  sanity.  I’ve  selected  a few  names  from  a 
very  long  list.  Regardless  of  which  platfonn  you  prefer,  the  compontents  to  create  and 
generate  a podcast  are  the  same.  Later  on  in  the  “Resources”  section  of  this  paper,  you’ll 
find  a larger  selection  of  technology  available. 

Podcasts  cover  a very  broad  spectrum  of  topics.  Since  there  are  no  Federal  Regulations 
to  oblige  by,  the  Podcaster  (the  emcee  whose  voice  you  hear  on  the  podcast),  does  not 
need  to  file  for  a license,  unlike  traditional  radio,  which  must  adhere  to  guidelines  set 
forth  by  the  government. 

As  a direct  result  for  not  having  a license,  anyone  can  generate  a podcast  and  therefore 
there  are  no  restrictions  to  what  topics  are  discussed  nor  how  “clean”  the  language  may 
be. 
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And  unlike  traditional  radio,  once  your  podcast  is  available  on  the  web,  it  is  then  part  of  a 
“global”  market  as  opposed  to  a “regional”  one.  This  means  that  you  listening  audience 
can,  literally,  span  the  entire  world  and  not  just  a small  local  geographic  area. 

Podcasts  are,  by  and  far,  FREE  - but  there  is  a movement  to  make  some  podcasts 
accessible  only  for  members  who  shell  out  fees.  An  example  of  this  is  slowly  making  its 
way  through  the  music  industry. 

Definition  of  an  “enclosure”  can  be  found  here: 
http://en.wikipedia.org/wiki/RSS  enclosure 

Industry  and  Podcasts 

Many  industries  have  launched  podcasts  to  keep  people  informed  of  any  new 
developments,  news  or  updates  to  existing  issues.  And  although  many  of  these  podcasts 
are  free  to  download  and  listen  to,  there  is  some  degree  of  work  and  planning  involved  in 
creating  a podcast  that  people  will  “tune-in”  for  on  a regular  basis. 

The  music  industry  has  helped  podcasting  immensely!  Independent  producers  and 
unsigned  artists  are  using  podcast  technology  to  reach  the  masses.  By  the  letter  of  the 
law,  P2P  (peer-to-peer)  services  are  legal  and  podcasts  are  free.  It’s  no  wonder  groups 
such  as  the  Artie  Monkeys  or  Gnarls  Barkley  have  reached  superstardom  in  the 
mainstream  music  world,  simply  based  on  the  number  of  times  their  music  has  been 
downloaded. 

Usually  podcasts  involve  music.  But  chances  are  the  music  you  find  will  not  be  from  any 
artist  you  may  recognize.  The  music  comes  from  unsigned  artists  who  are  looking  to 
break  into  the  industry. 

There  is  a “buzz”  to  make  podcasts  with  artist  we  all  recognize.  But  to  be  able  to  listen  to 
such  artists  would  require  us  to  A)  become  a member/subscriber  to  an  online  services  or 
B)  pay-as-we-go.  Understandable  since  the  music  is  of  course  copy  written  and  both  the 
artists  and  their  labels  would  need  to  be  paid. 

Other  industries  are  also  pushing  and  promoting  podcasts.  Sports  are  a major  promoter 
for  podcasts.  Take  a listen  to  ESPN  Radio  from  their  website.  There  you’ll  have  a 
choice  to  either  “play”  the  audio,  “download”  it  to  your  computer  or  go  to  i-Tunes  to 
send  it  to  your  iPod.  The  news  can  now  be  downloaded  as  a podcast.  Listen  to  CNN  and 
get  the  latest  information  or  download  a “video  podcast”  also  know  as  a “vodcast”  and 
watch  the  news  or  a CNN  program  such  as  Glenn  Beck.  Auto  maker  BMW  offers 
podcasts  off  its  website.  IBM  is  another  company  offering  podcasts  to  their  customers. 
These  industries  are  utilizing  the  same  technology  that  was  mentioned  earlier  and  are 
targeting  customers  and  the  general  public-at-large.  Now  education  is  following  this 
trend  to  target  students,  fellow  instructors,  parents  and  the  communities  they  serve. 
Below  are  the  various  URLs  referred  to  above: 
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ESPN  Radio  - http://espnradio.espn.go.com/espnradio/podcast/index 
CNN  - http://www.cnn.com/services/podcasting/ 

BMW  - http://podcast.bmw.com/en 

IBM  - http://www.ibm.com/investor/viewpoint/podcast/05-08-05-l.phtml 

Education  and  Podcasts 

Imagine  if  you  could  download  a podcast  to  help  you  learn  how  to  properly  pronounce 
words  in  a different  language  or  learn  English  as  a second  language.  Or  perhaps  find  a 
podcast  where  an  interview  with  a WWII  veteran  offers  you  a personal  point-of-view  of 
the  events  that  occurred  on  December  7,  1941.  You  might  find  a podcast  involving  a 
disscussion  on  diversity  issues  and  the  media.  These  are  examples  of  “podcasts  from  the 
classroom.” 

It’s  true  that  there  are  no  limits  to  the  topics  podcasts  cover  and  in  education  that  is  also 
the  case.  As  an  instructor,  one  can  opt  to  have  his/her  lectures  available  for  downloading 
to  a digital  media  player,  such  as  an  iPod,  Creative  Zen  or  the  new  Microsoft  Zune  to  be 
listened  to  for  reviewing  purposes.  Science  experiments  and  field  trips  can  generate 
discussions  that  could  make  great  podcasts,  which  can  then  generate  further  discussions 
outside  of  the  classroom  after  downloading  and  listening  to  them. 

Podcasting  has  been  around  for  a few  years  and  yet  is  still  considered  “new”.  Education 
doesn’t  end  at  institutions  but  has  expanded  to  include  Corporate  Training.  In  areas  such 
as  Graphic  Design  and  Multimedia,  you  can  download  either  a podcast  for  listening  or  a 
vodcast  to  see  how  various  techniques  are  achieved. 

By  providing  the  fuel  to  generate  creative,  intellectual  and  worthwhile  podcasts  for 
various  fields  of  study,  you  can  then  use  podcasts  in  developing  countries  where 
technology  and  education  are  now  beginning  to  take  form  shaping  the  country’s 
infrastructure. 

Using  podcasts  in  the  classroom  can  give  you  a first  person  point-of-view  of  what  is 
happening  in  “hot  spots”  around  the  world.  Listen  in  on  how  things  in  Darfur  are 
progressing  from  those  who  are  there  trying  to  make  a difference.  Or,  listen  to  podcasts 
from  both  sides  of  the  Israeli  and  Palestinian  conflict  in  the  Middle-East.  These  sorts  of 
podcast  are  created  by  people  who  live  in  a world  where  violence  and  conflict  have 
become  a way  of  life.  The  news  that  we  listen  to  may  have  a biased  and/or  scewed 
perspective  of  the  situation  and  it’s  usually  a good  idea  to  get  another  perspective. 
Podcasts  allow  you  the  opportunity  to  hear  various  sides  of  an  issue  which  allows  you  to 
formulate  your  own  opinion  of  a situation.  Below  are  some  URLs  that  illustrate  the 
examples  listed  above: 

Medicins  Sans  Frontieres  - http://www.msf.org/ 
iPod  - http://www.apple.com/itunes/ 

Creative  Zen  - http://www.creative.com/ 

Microsoft  ZUNE  (article  from  Podcasting  News)  - 
http://www.podcastingnews.com/2006/09/14/microsoft-zune-first-look/ 
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Conclusion 


Although  podcasting  is  a relatively  new  media,  newer  forms  are  already  making  their 
way  to  a digital  media  player  near  you.  These  newer  forms  are  known  as  “enhanced 
podcasts.”  They  are  designed  to  deliver  photos  and  other  images  in  synchronization  with 
the  audio  of  the  podcast  much  like  a slide  show.  As  you  can  imagine,  the  file  size  will  be 
much  larger  than  a standard  podcast  and  therefore  may  take  longer  to  download.  Below 
are  URLs  where  you  can  find  information  on  enhanced  podcasts: 

MakeZine  - http://www.makezine.com/extras/43.html 

Digital  Producer  - http://www.digitalproducer.com/articles/viewarticle.isp?id=37839 

Students  benefit  from  podcasts  in  many  ways.  Not  only  can  they  download  music  legally 
to  share  with  friends,  but  can  learn  about  the  latest  business  trends,  or  leam  how  to 
market  their  skills  in  an  ever-changing  economy,  or  listen  in  on  a first  person  point-of- 
view  from  someone  who  has  been  on  the  front  lines  battling  the  insurgents  in  Iraq. 

Podcasting  is  an  experience  that  is  fresh,  current,  entertaining,  infonnative  and  in  many 
cases,  insightful.  Where  a moment  in  time  is  captured  for  the  world  to  hear  and  share  and 
become  a part  of  human  history. 

Below  are  various  lists  of  resources.  These  lists  include  infonnation  on  how-to  create 
podcasts  for  both  PC  and  Mac;  the  sort  of  technology  needed  to  develop  a podcast,  along 
with  some  free  downloads,  and  where  to  upload  it  once  it’s  done.  There  is  infonnation 
on  books  that  can  be  purchased  from  your  local  bookstore,  as  well  as  online  resources 
and  directories  which  can  provide  you  with  many  useful  examples  of  podcasts. 


From  the  user  side: 

-Where  to  find  podcasts  on  various  topics? 
www.acidplanet.com/ 

http://www.virginradio.co.uk/thestation/podcasts/index.html 
http://www.npr.org/rss/podcast/podcast  detail.php?siteld=48 19382 

Samples  for  podcasts 

-Music,  Business  & Finance,  Technology,  News,  Educational,  etc 
(Demonstration  of  examples  of  various  podcasts.) 

University  of  Wisconsin:  http:// engage . doit. wise . edu/podcasting/teachAndLeam/ 
Librivox:  http://www.librivox.org/ 

Podcast  and  Portable  Media  Expo:  http://www.portablemediaexpo.com/audio.htm 
NPR:  http://www.npr.org/rss/podcast/podcast  directory.php 
Media  Artist  Secrets:  http://www.finstudio.com/blog/index.html 
Podcast  from  the  West  Bank  in  the  Gaza  Strip:  http://www.rabble.ca/rpn/btw/ 
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From  the  creators  side: 

-Tools  of  the  trade 

-Microphones  / Flash  Drive  / Digital  Recorders 

iRiver  (www . iriverameric a . c om) , iTalk  (for  iPods  www.apple.com) 

-Free  Software  vs.  Consumer  Software  vs.  Professional  Software 
-Audacity  (http://audacity.sourceforge.net/download/) 

-Adobe  Audition  (http://www.adobe.com/products/audition/main.html) 
-Sony  ACID 

(http://www.sonymediasoftware.com/products/acidfamilv.asp) 
-GarageBand  (http://www.apple.com/ilife/garageband/) 

-Soundtrack  (http://www.apple.com/Fmalcutexpress/soundtrack.html) 

-Generating  Enclosures 

-MP3  Tag  Tools  (http://sourceforge.net/proiects/massid31ib) 

-Podifier  (http://www.podifier.com/) 

-Where  to  host  your  podcast? 

- Our  Media  (http://www.ourmedia.org/) 

- Gcast  (http://www.gcast.com/7nrM &&s=60900592) 

- CastPost  (http://www.castpost.com/) 

- SwitchPod  (http://www.switchpod.com/) 

- iTunes  (http://www.apple.com/itunes/) 

- SlipCast  (http://slapcast.com/) 

- eSnips  (http://www.esnips.com/signin/index.isp) 

- ShockPod  (http://www.shockpod.com/) 

-Can  you  upload  to  your  existing  website? 

Yes,  you  can!  The  one  drawback  to  this  is  that  unless  you  have  a method  of 
monitoring  the  number  of  times  it’s  been  downloaded,  you’ll  never  know  how  well  your 
podcast  is  doing.  Monitoring  how  many  times  your  website  has  been  visited  does  not 
mean  your  podcast  has  been  downloaded  either. 

-What  is  an  Aggregator  / News  Reader? 

- As  defined  by  Google: 

http://www.  google.  com/scarch?hl=cn&q=dcfinc%3  A+Ncws+ Aggregators 

- As  defined  by  Wikipedia: 
http://en.wikipedia.org/wiki/News  aggregator 


- List  of  Aggregators  (from  Wikipedia) 
http://en.wikipedia.org/wiki/List  of  news  aggregators 

Copyright  Issues 

-Creative  Commons  - http://creativecommons.org/ 
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For  additional  resources  on: 

Creative  Commons 
Intellectual  Property 
Fair  Use  Act 

Digital  Millennium  Copyright  Act 
See  this  webpage  for  additional  websites: 

http://facultv.plattsburgh.edu/edwin.vega/copyright  information.htm 

Applications  where  podcasts  can  be  beneficial: 

-Books  on  Tape/CD 
-Review  lessons 
-Conduct  Interviews 
-Archiving  & Documentation 
-Learn  a new  language 
-TOD  - Teaching-On-Demand 

Who  is  currently  using  podcasts? 

Companies/Corporations 

-CNN  - http://www.cnn.com/services/podcasting/ 

-Podcast  and  Portable  Media  Expo  - 

http://www.portablemediaexpo.com/audio.htm 

-Disney  - http://disnev.go.com/music/podcasts/today/index.html 

-NPR  - http://www.npr.org/rss/podcast/podcast  directorv.php 

-ABC  News  - http ://abcnews . go . com/T echnology/Podcasting/ 

Education 

- Education  Podcast  Network  - http : //epnweb . or g/ 

- San  Francisco  Museum  of  Modern  Art  - 
http://www.sfmoma.org/education/edu  podcasts.html 

- Apple  - http://www.apple.com/education/podcasting/ 

- Ball  State  University  - 

http://www.bsu.edU/library/article/0, 1 894, 163773-11770-35591  .OO.html 

- The  Newsletter  of  the  National  Institute  for  Technology  and  Liberal  Education 
http://newsletter.nitle.org/v4  n3  summer2005/news.php 

(Scroll  down  to  the  middle  of  the  page:  Article  by  Bryan  Alexander) 

Training 

-Media  Artist  Secrets  - 
http://www.fmstudio.com/blog/index.html 


What’s  next? 

-VODcasting  - Video-On-Demand 
- Vlogging  - Video  Blogging 
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-Moblogging  - Mobile  Blogging 
-VCasts  - Video  Casts 

List  of  resources: 

Adam  Curry  - http://www.curry.com/ 

(One  of  the  godfathers  of  podcasting.) 

For  Mac: 

http://www.engadget.com/2004/10/05/engadget-podcast-001- 10-05-2004-how-to- 
podcasting-get/  - Engadget 

http://www.stager.org/podcasting.html  - Gary  S.  Stager 
http://macs.about.eom/b/a/183613.htm  - About.com 


For  PC: 

Educational  Podcasts: 

http://www.bsu.edU/library/article/0, 1 894, 163773-11770-35591  .OO.html  - The  Principles 
and  Methods  of  Digital  Listening:  Educational  Podcasting 

http://www.podcastingnews.com/articles/Make  Podcast  Blogger.html  - Making  a 
Podcast  with  Blogger  and  FeedBurner 

http://blog.wolverhamptonclc.co.uk/2005/08/09/podcasting-a-new-force-in-education/  - 
Podcasting:  A new  force  in  education 
http://epnweb.org/  - Education  Podcast  Network 
http://www.apple.com/education/ipod/  - iPod  in  The  Classroom 
http://www.macdevcenter.eom/pub/a/mac/2005/01/25/podcast.html  - How  to  record  a 
Podcast  (on  a Mac) 
http://edupodder.com/  - EduPodder 


Hosting: 

http://blipmedia.org/su/rp/?s=goog  (r.Podcaster) 

http://www.podcastingnews.com/archives/2005/08/new  podcast  vid.html  (Podcasting 
News) 

http://www.podbus.com/  (Podbus) 

http://creativecommons.org/tools/ccpublisher  (CC  Publisher) 
http://digitahnedia.oreillv.com/2005/09/07/hosting-podcasts.html  (O’Reilly) 
http://radio.indvmedia.org/  (Radio  Indy  Media) 
http://www.podcastdirectorv.com/add/  (Podcast  Directory) 

http://www.feedburner.eom/fb/a/popup-quickstart-podcast  (Publish  Your  Podcast) 
http://www.podcast.net/addpodcast  (Podcast) 

http://www.podcastallev.com/podcast  genres.php?pod  genre  id=3  (Podcast  Alley) 
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Podcast  Software: 

http://www.podifier.com/  (Podifier) 

http://ipodder.sourceforge.net/index.php  (iPodder) 

http://ipodderx.com/  (iPodderX) 

http://jpodder.com/  (JPodder) 

http ://www. apple. com/itunes/hp/ download/  (iTunes) 

http://www.apple.com/itunes/  (iTunes) 

http://podfeeder.com/  (PodFeeder) 

http://www.dorada.co.uk/  (RSS  Radio) 

http://www.dopplerradio.net/  (Doppler  Radio) 

http://thirstycrow.net/happyfish/default.aspx  (Happy  Fish) 

http://www.download3000.com/download  10509.html  (Podspider) 

Podcast  Info: 

http ://www.bbc . co .uk/radio/ downloadtrial/  (BBC  Radio) 
http://en.wikipedia.org/wiki/Podcasting  (Wikipedia) 
http://www.podcastingnews.com/  (Podcasting  News) 
http://www.podcast4 1 1 .com/  (Podcast  41 1) 
http://www.answers.com/topic/podcasting  (Answers.com) 

Tools: 

http://www.podcasting-tools.com/  (Resource  for  Podcasting) 
http://reviews.cnet.com/4520-645Q  7-6212319-l.html  (MP3  Insider) 
http://www.feedforall.com/tools-for-podcasting.htm  (Feed  For  All) 
http://www.e-learningcentre.co.uk/eclipse/vendors/podcasting.htm  (e-Learning  Centre) 
http://www.rss-specifications.com/podcasting-explained.htm  (Tools  for  Webmasters) 
http://www.garageband.com/podcast  (GarageBand.com) 

http://www.workz.com/content/view  content.html?section  id=464&content  id— 6834 
(WorkZ) 

http://www.potionfactorv.com/news.html  (Potion  Factory)  [Mac  Only] 
http://mixcastlive.com/category/podcasting-tools/  (MixCast  Live)  [Free  Download] 
http://www.apple.com/support/garageband/podcasts/  (from  Apple  for  everyone) 
http://www.podcastingnews.com/archives/2005/02/feeddemon  takes.html  (Podcasting 
News) 

http://www.acidplanet.com/podcasts/default.asp?p=Create&T=l  (ACIDplanet.com) 

Forums: 

http://creativecalfcreativecow.net/cgi-bin/new  view  posts,  cgi?  forum  id=205  (Creative 
Cow) 
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article:  libraries  turning  to  iPods  and  iTunes 


http://keillorxichriiond.edu/bloisoni/blog/ats/2005/09/07/Educational-Uses-of- 

Podcasting.html 


teaching  students  English/Japanese  through  podcasting 
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Software 


Proof  of  identity  required  - i.e.  Faculty,  Staff  or  Student  (ID  Cards  or  Transcripts) 

Academic  Super  Store  - http://www.academicsuperstore.com/ 

Studica  - http://www.studica.com/products/index.cfm 
Journey  Ed  - http://www.iourneyed.com/home.asp 
Advanced  Academic  Solutions  - http://www.advancedacademic.com/ 
Campus  Tech  - http://www.campustech.com/ 

Software  Express  - http://www.swexpress.com/ 

Creation  Engine  - http://www.creationengine.com/ 


Free  Loops  for  ACID  (PC)  and/or  GarageBand  (Mac):  (No  ID  required.) 

Free  ACID  Loops  - http://www.freeacidloops.net/ 

ACID  Fanatic  (download  the  previews  - they’re  FREE)  - 
http://www.acidranatic.com/index.php7sectionNoops 

ACIDplanet  (Home  of  the  8-Pack  every  Tuesday)  - 
http://www.acidplanet.com/tools/8packs/?t=8973 

Looperman  - http://www.loopennan.com/loops  samples  menu.php 

Loops  (Right-Click  to  save  WAV  files.)  - http://www.loops.net/ 

PlatinumLoops  - http://www.platinumloops.com/free  loops.shtml 

Indica  Audio  (Membership  Required  - it’s  FREE)  - 
http  ://www.  indicaaudio . com/  actions/home,  do 

The  Daily  WAV  (Soundbytes  from  TV  & Movies)  - 
http://dailvwav.com/index.php 

Free  Loops  - http://www.freeloops.com/ 

Future  Wave  Shaper  - http://www.futurewaveshaper.com/ 

Loop  Masters  (Can  download  Demo  MP3s)  - http://www.loopmasters.com/ 

PocketFuel  - http://www.pocketfiiel.com/gallery.php 

FindSounds  (Type  in  the  instrument  you’re  looking  for.)  - 
http://www.fmdsounds.com/ISAPI/search.dll 
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Nature  Sound  Effects  (Thunderstorms,  etc.)  - 
http://nature-downloads.naturesounds.ca/ 

Sound  Effects  - http://www.koumis.com/soundfx.htm 

SonnyBoo  (These  are  ZIP  files  of  INDIE  music)  - 
http://www.sonnyboo.com/music/music.htm 

Loop  Galaxy  (Membership  Required  - it’s  FREE)  - 
http://www.loopgalaxy.com/audio/user/signup.do 


Books: 

Podcasting  Now!  Audio  Your  Way 

by  Andrew  J.  Dagys 

• Paperback:  272  pages  ; Dimensions  (in  inches):  0.70  x 9.08  x 7.52 

• Publisher:  Course  Technology  PTR;  1 edition  (November  18,  2005) 
. ISBN:  1598630768 

Podcasting:  Do  It  Yourself  Guide 

by  Todd  Cochrane 

• Paperback:  298  pages  ; Dimensions  (in  inches):  0.73  x 9.28  x 7.44 

• Publisher:  Wiley;  (June  10,  2005) 

• ISBN:  0764597787 

Podcast  Solutions:  The  Complete  Guide  to  Podcasting  (Solutions) 

by  Michael  Geoghegan,  Dan  Klass 

• Paperback:  240  pages  ; Dimensions  (in  inches):  0.69  x 9.26  x 7.44 

• Publisher:  friends  of  ED;  Bk&CD-Rom  edition  (August  22,  2005) 

. ISBN:  1590595548 

Secrets  of  Podcasting  : Audio  Blogging  for  the  Masses 

by  Bart  G.  Farkas 

• Paperback:  224  pages  ; Dimensions  (in  inches):  0.46  x 9.04  x 7.08 

• Publisher:  Peachpit  Press;  (August  9,  2005) 

. ISBN:  0321369297 
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Absolute  Beginner's  Guide  to  Podcasting  (Absolute  Beginner's  Guide) 

by  George  Colombo,  Curtis  Franklin 


• Paperback:  264  pages  ; Dimensions  (in  inches):  0.64  x 9.02  x 7.40 

• Publisher:  Que;  (October  17,  2005) 

• ISBN:  0789734559 

Tricks  of  the  Podcasting  Masters 

by  Robert  Walch,  Mur  Lafferty 

• Paperback:  360  pages 

• Publisher:  Que;  (May  25,  2006) 

• ISBN:  0789735741 

Hands-On  Guide  to  Video  Blogging  and  Podcasting,  First  Edition  : Emerging  Media 
Standards  for  Business  Communication  (Hands-On  Guide  Series) 

by  Damien  Stolarz,  Lionel  Felix 

• Paperback:  352  pages 

• Publisher:  Focal  Press;  (April  17,  2006) 

. ISBN:  0240808312 

Podcasting  For  Dummies  ® (For  Dummies  (Computer/Tech)) 

by  Tee  Morris,  Evo  Terra,  Dawn  Miceli,  Drew  Domkus 

• Paperback:  360  pages  ; Dimensions  (in  inches):  0.80  x 9.20  x 7.36 

• Publisher:  For  Dummies;  (November  21,  2005) 

. ISBN:  0471748986 
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For  additional  information  on  Podcasts  and  Aggregators,  please  visit  Edwin  Vega’s 
Faculty  Website  at  www.plattsburgh.edu  and  look  for  “Podcasting  Information”  and 
“Aggregators  / Readers”: 

Podcasting  - http://faculty.plattsburgh.edu/edwin.vega/psx  podcasting.htm 
Aggregators  - http://faculty.plattsburgh.edu/edwin.vega/psx  aggregators.htm 


To  view  how  podcasting  is  currently  being  used  in  the  classroom,  please  visit  Darcy 
Aubrey’s  Website  at  http://student.plattsburgh.edu/  and  look  for  her  WebQuest: 

WebQuest  - http://student.plattsburgh.edu/aubrl818/podcastingppt  files/frame.htm 
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Procedural  Principles  of  Design 

The  Role  of  Analysis  and  Evaluation  in  Educational  Design  Processes 

Irene  Visscher-Voennan 

Summary 

Educational  designers  conduct  a variety  of  design  activities  in  practice.  Building  on  several  reconstructive 
studies  of  design  practice,  8 procedural  design  principles,  related  to  analysis  and  formative  evaluation  activities  were 
formulated  and  discussed  and  elaborated  in  a one-day  workshop  with  24  educational  designers  from  various 
contexts.  Purpose  was  to  check  the  extent  to  which  these  designers  thought  these  principles  were  valid  for  their  own 
situation,  and  to  increase  the  robustness  of  the  principles,  by  further  specifying  and  contextualizing  them.  The  results 
of  the  workshop  show  that  the  amount  of  innovation  of  the  product  to  be  designed  impacts  the  way  in  which  analysis 
and  evaluation  activities  are  conducted.  Results  also  show  that  analysis  and  evaluation  strategies  differ  in  education 
versus  training  contexts,  or  in  generic  versus  specific  situations. 

Introduction 

The  work  of  designers  in  the  field  of  instructional  design  and  educational  technology  is  aimed  at  improving 
learning  and  performance  in  a variety  of  settings  (Reiser  & Dempsey,  2006).  In  their  design  processes,  these 
educational  designers  are  assumed  to  work  systematically  while  analyzing  learning  and  performance  problems,  and 
designing,  developing,  implementing  and  evaluating  instructional  and  non-instructional  processes  and  resources.  In 
literature,  these  types  of  activities  have  been  visualized  in  so-called  ADDIE-models  with  a more  or  less  cyclic 
nature.  The  last  two  decades,  there  has  been  increased  awareness  that  designers  may  work  differently  from  what  is 
proposed  by  traditional  ADDIE-design  models.  This  has  resulted  in  a number  of  reconstructive  studies,  in  which 
design  practice  has  been  studied,  resulting  in  practical  design  knowledge.  Not  only  were  designers  found  to  plan  less 
rigorous  and  systematic  strategies  to  meet  constraints  such  as  limited  time,  budget,  and  personnel,  they  were  also 
forced  to  adapt  their  approach  along  the  way,  because  of  unplanned  circumstances  (c.f.  Pieters  & Bergman,  1995; 
Walker,  1990;  Wedman  & Tessmer,  1993;  Winer  & Vazquez-Abad,  1995;  Visscher-Voerman  & Gustafson,  2004). 
This  results  in  tenable  shortcuts  and  in  selective  and  careful  (non)use  of  design  activities  contained  in  the  models. 
Clear  differences  between  practice  and  literature  come  to  the  fore  with  respect  to  analysis  and  formative  evaluation 
activities  and  their  alleged  function.  One  of  the  areas  where  the  gap  between  practice  and  literature  is  most  evident  is 
that  of  formative  evaluation,  either  or  not  in  relation  to  analysis. 

Most  reconstruction  studies  result  in  a summary  and  discussion  of  the  results  in  contrast  with  literature,  and 
eventually  provide  a reflection  on  the  worth  of  the  strategies  or  activities  applied  for  the  studied  contexts  and  for 
other  contexts.  As  such,  it  yields  articulated  state-of-the-art  knowledge  about  design  practice.  If  one  wants  to  help 
developing  design  methodology,  it  is  also  important  to  bring  the  findings  to  a next  level.  Firstly,  by  identifying  the 
activities  that  are  judged  relevant  and  useful  in  addition  to  existing  methods,  and  that  could  have  potential  for  other 
designers  in  similar  or  other  contexts  as  well,  formulating  them  as  procedural  design  principles.  Secondly,  by 
exploring  to  what  extent  the  principles  are  valid  for  other  contexts,  situations  and  designers.  And  thirdly,  by  further 
increasing  the  robustness  of  the  principles  by  contextualizing  them,  i.e.  by  specifying  more  precisely  in  what 
situations  these  strategies  are  valid. 

These  steps  were  followed  in  a study  on  which  this  paper  reports.  Eight  procedural  design  principles, 
concentrated  on  educational  analysis  and  evaluation  activities,  were  formulated  based  on  findings  in  literature  and 
reconstructive  studies.  In  a one-day  workshop  with  24  designers  from  various  design  contexts,  these  principles  were 
then  validated  and  further  contextualized.  The  general  question  to  be  answered  is  ‘What  are  practical  and  robust 
procedural  design  principles,  related  to  educational  analysis  and  evaluation?’  The  study  was  a follow-up  study  of  the 
DATE-research  (Visscher-Voerman,  1999;  Visscher-Voerman  & Gustafson,  2004)  in  which  24  design  practices 
were  reconstructed.  This  paper  introduces  the  design  principles,  describes  the  design  of  the  workshop  and  presents 
and  discusses  the  results.  The  paper  will  also  reflect  briefly  on  the  worth  and  function  of  such  a workshop  as  a 
particular  research  activity  in  a process  of  formulating  and  increasing  design  knowledge. 
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Towards  The  Formulation  of  Procedural  Design  Principles 


The  first  step  in  the  study  was  to  formulate  procedural  design  principles.  In  order  to  be  able  to  formulate 
valuable  improvements  to  existing  design  methods,  the  focus  was  on  those  aspects  that  showed  a tension  or  gap 
between  the  assumptions  and  starting  points  of  traditional  ADDIE-models  as  studied  in  literature,  and  current 
practices.  Also,  those  strategies  were  selected  that  seemed  to  be  promising  and  useful  ways  of  dealing  with  typical 
design  dilemmas.  One  of  the  areas  where  the  gap  between  practice  and  literature  was  most  evident  is  that  of 
formative  evaluation,  either  or  not  in  relation  to  analysis.  In  this  area,  8 procedural  design  principles  were 
formulated  and  discussed  in  the  workshop.  The  subsequent  sections  will  name  the  8 principles  as  used  in  the 
workshop.  For  each  principle,  a brief  rationale  will  be  given  for  its  content  and  why  this  principle  was  chosen.  At  the 
end  of  this  section.  Table  1 presents  an  overview  of  all  principles. 

Design  As  A Solution-Driven  Process  And  The  Role  of  Analysis 

In  traditional  ADDIE  models,  the  analysis  phase,  as  the  first  phase  in  the  process,  is  meant  to  explore  the 
problem  and  its  context,  usually  resulting  in  a problem  definition.  It  includes  the  identification  of  the  user  or  learner 
group,  analysis  of  the  tasks  and  skills  to  be  mastered  or  content  to  be  learned,  demarcating  the  user  environment 
(e.g.  Kessels,  1993;  Romiszowksi,  1981;  Smith  & Ragan,  2004).  The  analysis  should  provide  the  designer  with 
enough  information  and  insights  in  the  situation  that  (s)he  is  able  to  further  plan  the  process. 

One  conclusion,  derived  from  several  reconstructive  studies  (cf.  Rowland  & Wilson,  1994;  Schon,  1983, 
Visscher-Voerman  & Gustafson,  2004,  Walker,  1990)  is  that  the  nature  of  design  processes  is  solution-driven,  rather 
than  problem-driven.  Rather  than  conducting  a full  analysis,  designers  conduct  only  a restricted  analysis,  which  is 
also  more  focused  on  developing  solution  specifications  than  on  the  problem.  To  stimulate  solution  specification, 
designers  conduct  several  analysis  activities  that  seem  to  be  useful  and  cost-effective:  the  making  of  early 
prototypes;  reviewing  existing  materials;  showing  products  from  former  projects.  Despite  the  chance  of  premature 
closure  of  the  solution,  such  activities  help  to  speed  up  the  process.  Of  course,  it  seems  logical  that  designers  should 
reflect  critically  on  those  solution  options.  A careful  consideration  of  how  those  solutions  will  fit  the  current 
situation,  and  a consideration  of  potential  pitfalls  and  problems  to  be  expected,  will  help  to  create  a more  realistic 
view  of  the  design  process  to  be  planned.  These  considerations  resulted  in  the  formulation  of  three  design  principles 
(principles  1,  2,  3). 

For  many  designers,  the  consideration  of  alternative  solutions  is  likely  to  be  an  essential  part  of  the  design 
process.  Even  if  designers  have  clear  ideas  for  the  (potential)  solution  at  the  start  of  the  process,  the  DATE-study 
(Visscher-Voerman  & Gustafson,  2004)  showed  that  a majority  of  designers  considered  alternative  solutions  as  an 
essential  part  of  the  design  process.  Principle  4 has  been  formulated  so  as  to  check  participants’  opinion  on  this 
point. 

Because  of  the  large  amount  of  uncertainty,  which  is  a characteristic  of  the  majority  of  design  situations, 
the  following  tactic  seems  to  be  very  useful.  It  seems  reasonable  that  designers  split  the  design  process  into  phases 
with  formal  decision  moments  and  concrete  products,  and  should  only  plan  the  upcoming  phase  in  detail  (principle 

5). 

The  Role  Of  Formative  Evaluation 

The  evaluation  of  a design  under  construction  is  a distinct  phase  in  the  ADDIE  models.  Through  evaluation 
activities,  the  quality  of  the  product  under  development  is  being  assessed.  In  accordance  with  the  findings,  the 
product  may  either  be  improved  or  implemented  as  it  is.  A typical  evaluation  activity  is  the  pilot  test  or  try-out  of  the 
product  late  in  its  development  (e.g.  Andrews  & Goodson,  1991).  However,  several  authors  and  researchers  have 
stressed  the  importance  of  formative  evaluation  and  advice  to  conduct  evaluative  activities  early  in  the  process, 
integrated  with  design  activities  (e.g.  Tessmer,  1994,  Nieveen,  1997).  The  survey  research  by  Winer  and  Vazquez- 
Abad  (1995)  showed  that  instructional  designers  increasingly  acknowledge  the  worth  of  formative  evaluation  (pilot 
testing),  and  increasingly  evaluate  several  prototypes  of  a product  before  they  deliver  the  final  product.  Others, 
however,  found  that  evaluation  activities  do  not  receive  the  amount  of  time  and  attention  as  suggested  in  traditional 
design  models  (Van  den  Akker  et  al.,  1990;  Pieters  & Bergman,  1995,  Wedman  & Tessmer,  1993,  Zemke,  1985;  see 
also  Keursten,  1994;  Leshin,  Polluck  & Reigeluth,  1992).  The  results  from  the  DATE-study  (Visscher-Voerman  & 
Gustafson,  2004)  do  not  fully  support  this  conclusion.  Here,  all  designers  deliberately  conducted  one  or  more 
evaluation  activities,  often  interwoven  with  design  activities.  This  shows  that  designers  do  not  only  perceive  the 
value  of  formative  evaluation,  but  also  try  to  integrate  these  activities  in  their  processes.  Yet,  it  was  also  noted  that 
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only  in  a few  cases  designers  conducted  their  evaluations  systematically,  according  to  previously  formulated  plans 
and  criteria.  Most  evaluations  were  conducted  informally,  relying  on  personal  judgment  of  designers  and 
stakeholders  involved.  Designers  also  indicated  relying  on  small  numbers  of  evaluation  respondents  in  order  to  keep 
their  evaluations  efficient.  Although  this  seems  plausible,  given  the  constraints  of  many  design  projects,  designers 
could  run  the  risk  of  gathering  one-sided  or  biased  information.  If  designers  would  try  to  increase  triangulation 
(Miles  & Huberman,  1994)  over  persons  or  methods,  this  could  increase  their  effectiveness.  This  led  to  the 
formulation  of  principle  6. 

Another  obvious  finding  was  that  evaluations  were  not  only  used  to  check  the  quality  of  the  product  under 
development,  but  also  to  sharpen  the  design  specifications.  Some  designers  started  designing  as  early  as  possible. 

The  use  of  concrete  prototypes  was  seen  as  an  essential  means  for  identifying  the  specifications  of  the  product  in 
interaction  with  client,  experts,  and,  especially,  members  of  the  user  population.  Depending  on  user  satisfaction  and 
available  time  and  finances,  the  process  of  design,  evaluation,  and  revision  is  repeated  several  times.  This  is 
regarded  the  best  way  to  gather  user  insights,  needs,  and  wishes  regarding  the  validity,  practicality,  and  effectiveness 
(Nieveen,  1997)  of  the  products.  Evaluations  can  thus  play  an  important  role  in  clarifying  product  specifications 
(principle  7). 

Several  designers  regarded  it  useful  to  conduct  their  evaluations  themselves,  as  an  efficient  way  to  integrate 
evaluations  in  their  processes.  Despite  the  risk  of  designer  bias,  this  seems  a plausible  tactic,  especially  for  early 
evaluations,  where  the  focus  may  be  more  on  improving  the  design,  rather  than  on  determining  its  final  and  overall 
value.  This  led  to  the  formulation  of  the  principle  8. 

Table  1 : Procedural  design  principles  as  input  for  the  workshop 

1 . Designers  should  make  a prototype  in  an  early  stage  of  the  design  process. 

2.  A useful  means  to  help  clients,  partners,  and  other  stakeholders  to  choose  a solution  and  to  formulate 
product  specifications  is  by  showing  products  from  former  projects. 

3.  An  essential  part  of  the  analysis  phase  is  a consideration  of  possible  pitfalls  and  problems  during  the 
design  and  implementation  phases. 

4.  Even  if  designers  have  a clear  idea  for  the  (potential)  solution  at  the  start  of  the  process,  consideration  of 
possible  alternative  solutions  is  essential. 

5.  Designers  should  split  the  design  process  into  phases  with  formal  decision  moments  and  concrete 
products,  and  should  only  plan  the  upcoming  phase  in  detail. 

6.  For  efficient  and  effective  formative  evaluations,  several  (about  three)  sources  and  several  (about  three) 
data  gathering  instruments  should  be  used. 

7.  In  order  to  clarify  product  specifications,  designers  should  spend  their  time  on  carefully  planned  formative 
evaluations  of  early  versions  of  a prototype,  rather  than  on  an  elaborate  preliminary  analysis. 

8.  Designers  should  conduct  formative  evaluations  themselves. 


Design  Of  The  Workshop 

The  function  of  the  workshop  was  to  validate  the  design  principles  derived  from  the  reconstructive  studies 
and  to  increase  their  robustness.  The  principles  were  presented  to  24  designers  from  various  design  contexts,  in 
order  to  check  the  extent  to  which  these  designers  valued  these  principles  valid  for  their  own  situation,  and  to  further 
specify  them  to  different  design  contexts.  In  order  to  make  all  participants’  actions  visible  and  traceable,  all 
materials  to  be  used  in  the  workshop  were  provided  with  unique  codes  (one  for  each  designer).  Also,  the  whole  day 
was  videotaped,  so  that  comments  made  could  be  replayed.  The  workshop  activities  were  pilot  tested,  to  test  the 
procedure  as  well  as  the  clarity  of  the  principles  formulated,  with  eleven  researchers  from  the  department  of 
Curriculum  at  the  University  of  Twente,  most  of  them  having  experience  within  one  of  the  six  design  contexts.  Only 
minor  changes  were  made.  These  subsequent  sections  describe  and  account  for  the  participants  selected  and  the 
workshop  method  chosen. 

Participants 

As  participants  in  the  workshop,  the  choice  was  made  to  invite  both  designers  who  participated  earlier  in 
the  DATE-study  and  ‘new’  ones.  The  choice  to  ask  designers  who  had  been  interviewed  partly  relied  on  the  belief 
they  had  a right  to  know  and  might  be  interested  in  the  overall  findings  of  the  DATE-research  (cf.  Miles  & 
Huberman,  1994;  Stake,  1976).  The  reason  to  ask  new  designers  as  well,  was  to  check  the  validity  of  the  principles 
with  new  designers  and  design  situations.  Since  another  goal  of  the  workshop  was  to  contextualize  the  principles,  it 
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was  important  to  select  designers  from  different  contexts.  To  assure  sufficient  variety,  six  different  contexts  were 
distinguished  (see  table  2).  Ultimately,  twenty-four  designers  participated  in  the  workshop.  There  were  12  male  and 
12  female  designers.  Their  design  experience  ranged  from  2 to  21  years.  13  Designers  had  a formal  background  in 
design  education,  8 of  them  at  the  Faculty  of  Educational  Science  and  Technology  at  the  University  of  Twente.  1 1 
Designers  had  either  a background  in  the  subject  matter  area  or  in  other  studies,  mostly  within  the  educational 
domain,  such  as  general  educational  studies. 


Table  2:  Designers  participating  in  the  workshop  (n=24),  spread  over  6 contexts. 


Context 

Interviewed 

New 

Total 

TB 

Textbook  design 

2 

2 

4 

CD 

Curriculum  development 

1 

2 

3 

MM 

Design  of  educational  multimedia 

2 

2 

4 

IT 

Design  of  internal  training  and  HRD  programs 

2 

1 

3 

ET 

Design  of  training  and  HRD  programs  in  external  bureaus 

1 

6 

7 

DE 

Design  of  distance  education 

3 

0 

3 

Total  11 

13 

24 

Workshop  Activity 

For  the  workshop,  the  design  principles  were  put  on  flip-over  sheets,  which  were  spread  over  the  walls  in 
the  room.  During  the  workshop,  the  design  principles  were  discussed  and  elaborated  in  small-groups,  using 
metaplan  techniques  (Schnelle,  1978).  The  activity  consisted  of  four  steps.  The  first  two  steps  were  aimed  at 
validating  the  principles,  the  third  at  further  contextualizing  the  principles  in  small  groups,  the  fourth  at  validating 
the  contextualized  principles. 

Firstly,  review  of  principles  and  voting  on  hand-out  (30  minutes).  The  workshop  moderator  reviewed  each 
principle  briefly,  clarifying  the  meaning  of  words  and  explaining  the  tone.  Participants  were  asked  to  judge  the 
tenability  of  each  principle  in  their  own  specific  context.  For  each  principle,  they  were  asked  to  put  their  votes  on  a 
hand-out,  by  indicating  on  a 4-points  scale  to  what  extend  they  agree.  They  could  also  circle  the  question  mark  if 
they  would  not  know  yet.  With  the  first  few  principles,  designers  started  to  discuss  meaning  of  words.  In  any  of 
these  cases,  the  workshop  moderator  suggested  the  designers  to  save  those  remarks  and  considerations  for  the 
discussion  in  small  groups.  This  first  step  was  a preparation  for  the  next  step,  and  was  planned  to  guarantee  that 
designers  scored  individually,  and  were  not  influenced  by  scores  from  their  colleagues. 

Secondly,  voting  with  stickers.  In  this  10-minutes  step,  designers  were  asked  to  make  their  votes  visible  for 
others,  by  putting  a sticker  on  each  flip-over  sheet,  corresponding  to  their  votes  on  the  hand-out.  Everyone  having 
voted,  the  workshop  moderator  took  a few  minutes  to  go  over  each  principle  to  search  for  striking  patterns  (such  as 
much  agreement  or  disagreement  on  a principle).  Since  designers  from  different  design  contexts  received  different 
colored  stickers,  it  was  also  easily  traceable  whether  there  were  clear  differences  between  or  within  different  design 
contexts. 

Thirdly,  elaboration  of  principles  and  defining  heuristics  in  small  groups  (2  hours).  Designers  were  divided 
into  eight  groups  of  three  persons.  The  groups  were  formed  so  as  to  get  a variation  over  contexts  within  groups  as 
much  as  possible.  Each  group  was  provided  with  four  design  principles  to  work  on.  Thus,  each  principle  was  being 
discussed  by  two  groups.  Groups  were  instructed  a)  to  generate  arguments  in  favor  of  the  principle  and  to  state 
situations  or  contexts  in  which  they  believed  the  principle  is  valid;  b)  to  generate  arguments  against  the  principle  and 
to  state  situations  or  contexts  in  which  they  believed  the  principle  is  not  valid;  c)  to  reformulate  the  principles  into 
new  and  valid  ones,  making  use  of  the  generated  arguments.  After  Van  den  Akker  (1999),  these  new  principles 
should  be  formulated  using  the  following  format:  If  you  want  to  make  . . ..(product  specification). ...  for  context 
. . ..(context  specification). ...  then  you  should  do  ...  .(action).. . . because  . . ..(rationale).. . . Having  finished  the  group 
work,  designers  put  these  new  heuristics  on  paper  and  placed  these  next  to  the  old  principles  on  the  flap-over  sheets 
in  the  room.  Then,  each  designer  could  walk  through  the  room  and  read  what  other  groups  had  come  up  with. 

Fourthly,  plenary  review  and  discussion  of  new  principles.  During  this  45-minute  review,  the  workshop 
moderator  asked  each  group  to  share  their  most  important  point  of  discussion  or  most  critical  aspect. 

Results 

This  section  describes  the  results  of  the  workshop  activity.  First,  the  overall  scores  on  the  principles  will  be 
presented,  together  with  an  overview  of  the  general  pattern  in  scores.  Then,  the  scores  for  each  principle  will  be 
listed  and  discussed,  according  to  the  following  structure:  indication  of  overall  score,  table  with  scores  differentiated 
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to  the  six  design  contexts,  supplemented  with  the  explanation  given  by  the  workshop  moderator,  arguments  in  favor 
of  and  against  the  principle  as  generated  in  the  small  group  discussion,  and  the  new,  contextualized  design  heuristics 
as  the  result  of  the  discussion. 

General  Patterns 

Table  3 provides  an  overview  of  the  overall  scores  on  the  principles.  There  are  several  general  patterns: 

Overall,  designers  agreed  highly  on  the  different  principles,  and  most  principles  were  scored  positively  (‘+’  or 
‘++’); 

75%  or  more  of  the  designers  scored  positively  on  principles  1,  2,  3,  4,  5; 

Highest  agreement  between  designers  was  reached  on  principle  3; 

Designers  scored  most  heterogeneously  on  principles  6,  7,  and  8,  of  which  principle  7 was  scored  most 
negatively  (‘-‘  or 

Designers  scored  relatively  few  question  marks; 

Within  all  contexts,  the  DE-designers  scored  most  heterogeneously. 


Table  3:  Range  of  votes  for  each  principle 


Principle 

++ 

+ 

- 

? 

Principle 

++ 

+ 

- 

? 

1 

12 

10 

0 

1 

1 

5 

12 

6 

4 

2 

0 

2 

13 

10 

1 

0 

0 

6 

5 

6 

11 

1 

1 

3 

20 

4 

0 

0 

0 

7 

1 

6 

9 

6 

2 

4 

13 

8 

2 

1 

0 

8 

4 

8 

8 

3 

1 

Principle  1:  Designers  Should  Make  A Prototype  In  An  Early  Stage  Of  The  Design  Process 

Table  4 shows  the  scores  for  the  principle  as  well  as  the  explanation  provided  by  the  workshop  moderator. 
Overall,  designers  highly  agreed  on  this  principle,  arguing  that  early  examples  help  to  visualize  and  discuss  the 
desirable  outcome  of  the  project.  Only  one  TB-designer  disagreed  fully,  arguing  that  prototypes  are  cost-expensive, 
and  should  thus  be  avoided.  In  this  respect,  it  is  important  to  use  the  term  ‘prototype’  in  the  sense  as  formulated  in 
the  explanation.  Evaluations  of  ‘prototypes’  being  a 90%  version  of  the  final  product  may  indeed  be  cost-expensive, 
especially  when  major  changes  need  to  be  made. 


Table  4:  Scores  on  principle  1 


++  + 
TB  3 0~~ 

CD  2 1 

MM  2 2 

IT  2 1 

ET  2 5 

DE  _l 1 

12  10 


? 

0 I 0 
0 0 0 
0 0 0 
0 0 0 
0 0 0 
0 0 1 
0 - 1 1 


Explanation 

The  concept  of  ‘prototype’  refers  to  any  kind  of 
materialization  or  visualization  of  an  idea  for  a solution, 
or  part  of  the  solution  (e.g.  first  description  of  a product, 
outline  of  a chapter,  a framework,  etc.).  It  should  be 
stressed  that  designers  should  make  such  a prototype 
early  in  the  process. 


Arguments  in  favor 

+ Early  prototype  can  serve  as  vehicle  to  reduce 
uncertainties  and  clarify  design  specifications. 

+ In  case  of  innovative  situations  (such  as  multimedia 
design  or  innovative  learning  processes),  when 
clients  are  not  familiar  and  inexperienced. 

+ Prototype  evaluation  stimulates  discussion  between 
client  and  users,  and  might  increase  commitment 

New  design  heuristics 

■ In  case  of  an  innovative  or  a complex  product,  designers  should  make  an  early  prototype  (=concretizations  of 
(part  of)  a possible  solution),  because:  it  clarifies  the  (consequences  of  the)  design  specifications;  it  can  be 
tested  quickly;  and  it  increases  commitment. 

■ When  creating  a product  for  a situation  with  innovative  elements,  designers  should  develop  a prototype  in  an 
early  stage,  because  expectations  could  be  tuned  and  uncertainties  be  reduced. 


Arguments  against 

Not  always  cost-effective 

Not  in  case  of  clear  design  specifications,  available 
examples  or  duplicate  of  a former  product,  or  in 
case  of ‘standard’  processes. 
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Principle  2:  A Useful  Means  To  Help  Clients,  Partners,  And  Other  Stakeholders  To  Choose  A Solution  And 
To  Formulate  Product  Specifications  Is  By  Showing  Products  From  Former  Projects 

There  was  a strong  agreement  on  this  principle.  Only  one  ET-designer  scored  partly  negative  (Table  5), 
giving  a warning  that  designers  should  stay  open,  and  not  let  the  example  solution  fully  define  the  problem  to  be 
solved.  This  warning  is  integrated  in  the  first  new  design  heuristic.  It  also  opens  the  reasoning  for  principle  4. 
Overall,  participants  agreed  that,  comparable  to  principle  2,  this  activity  can  really  clear  up  matters  and  help  specify 
the  outcomes  of  the  project. 

The  new  heuristics  happen  to  be  formulated  for  the  context  of  curriculum  design  only.  The  formulation 
shows  that  designers  intend  to  show  solution  examples  from  adjacent  domains  (which  may  be  easier  to  empathize 
with)  and  that  rely  on  comparable  design  principles. 


Table  5:  Scores  on  principle  2 


++  + 

- 

? 

Explanation 

TB  2 2 

0 0 

0 

By  showing  one  or  more  products,  or  by  explaining 

CD  2 1 

0 0 

0 

what  you  have  done  in  former  projects,  you  help  the 

MM  2 2 

0 0 

0 

client  shape  his  ideas  and  needs. 

IT  0 3 

0 0 

0 

ET  5 1 

1 0 

0 

DE  2 1 

0 0 

0 

13  10 

1 0 

0 

Arguments  in  favor 

Arguments  against 

+ Cheap  and  inspiring  activity 

Risk  of  premature  closure,  of  having  the  example 

+ Clears  up  matters  and  questions  (leading  from 

solution  define  the  problem  to  be  solved. 

abstract  to  concrete  specifications) 

Can  create  negative  opinions  about  solution  chosen 

+ Do  not  only  show  solutions  but  also  design 

approach  and  its  potential  results 

New  design  heuristics 


■ When  designing  for  clients  who  are  unfamiliar  with  the  type  of  solution,  designers  should  show  them  a wide 
range  of  possible  solutions,  because  that  will  be  inspiring  and  illuminating.  But,  it  should  not  lead  to  copying  a 
solution  unthinkingly  or  to  insufficient  problem  analysis. 

■ If  you  want  to  design  course  materials  for  the  subject  French  to  be  taught  in  schools,  you  should  show  course 
materials  for  the  subject  English  that  are  based  on  the  same  principles,  because  that  provides  the  client  with  a 
concrete  picture  of  the  course  materials  and  the  results  gathered  with  it  thus  far. 

■ In  case  of  curriculum  design  for  a general  context,  designers  should  show  a concrete  example  of  a teaching 
method  in  a comparable  domain,  because  that  gives  the  client  a concrete  idea  of  the  teaching  method  used  and 
results  obtained. 

■ In  case  of  curriculum  design  for  a general  context,  designers  should  be  careful  with  showing  products  from 

former  projects,  because  the  example  could  come  to  lead  its  own  life  and  push  the  actual  problem  into  the 
background. 


Principle  3:  An  Essential  Part  Of  The  Analysis  Phase  Is  A Consideration  Of  Possible  Pitfalls  And  Problems 
During  The  Design  And  Implementation  Phases 

There  was  a very  strong  agreement  on  this  principle  (Table  6).  Only  four  designers,  from  different  sectors, 
only  agreed  partly.  The  other  designers  all  agreed  completely  with  the  principle.  The  remarks  made  seemed  to  focus 
just  on  the  importance  of  analysis,  per  se  as  resulting  in  a baseline  against  which  decisions  can  be  made  regarding 
the  product  to  be  designed.  An  analysis  focusing  on  needs  may  in  itself  prevent  later  problems  from  occurring. 

Both  new  design  heuristics  show  that  the  amount  in  which  designers  have  physical  access  to  stakeholders 
and  target  group  determines  whether  the  analysis  is  focused  on  ‘now’  (analysis  of  current  needs)  or  is  oriented  on 
the  future  (consider  possible  problems  in  future  phases).  Formulated  in  this  way,  the  difference  between  corporate 
programs  and  educational  programs  is  also  a difference  between  specific  and  generic  situations. 

Principle  4:  Even  If  Designers  Have  A Clear  Idea  For  The  (Potential)  Solution  At  The  Start  Of  The  Process, 
Consideration  Of  Possible  Alternative  Solutions  Is  Essential 

Twenty-one  designers  (88%)  scored  positively  on  this  principle  (see  Table  7).  Designers  from  the  sectors 
IT,  ET,  and  DE  were  most  consistent  in  their  scores.  In  the  TB,  CD,  and  MM  sector,  a small  minority  scored 
negatively  on  the  principle.  CD-designers  showed  the  largest  spread  in  scores.  Comparison  of  the  arguments  shows 
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that  the  arguments  in  favor  seem  to  stress  the  importance  of  the  principle  in  general,  whereas  the  arguments  against 
show  specific  situations  in  which  the  principle  should  not  be  applied.  Such  as  when  designers  need  or  want  to  design 
a specific  type  of  product  (because  it  is  being  subsidized  or  because  they  want  to  make  exemplary  materials).  Or  put 


Table  6:  Scores  on  principle  3 


++  + 

- 

? 

Explanation 

TB 

3 1 

0 

0 

0 

This  principle  refers  to  a consideration  of  certain 

CD 

3 0 

0 

0 

0 

possible  consequences  of  a specific  design.  This  could 

MM  4 0 

0 

0 

0 

lead  to  anticipation  of  this  problem  during  the  process, 

IT 

2 1 

0 

0 

0 

or  to  the  choice  not  to  go  that  direction. 

ET 

6 1 

0 

0 

0 

DE 

2 1 

0 

0 

0 

20  4 

0 

0 

0 

Arguments  in  favor 

Arguments  against 

+ 

In  analysis  phase,  designers  determine  what  clients 

Not  in  case  of  innovative  situations,  in  which 

want,  which  is  the  basis  for  the  formulation  of 

certain  vision  or  ideology  is  being  developed.  They 

learning  goals,  and  accordingly,  the  design. 

should  not  be  constrained  before  they  even  started. 

+ 

Important  to  check  whether  client’s  initial  solution 

ideas  fit  real  needs.  Will  prevent  problems  in 

implementation  phase. 

New  design  heuristics 


■ When  designing  a corporate  development  program,  designers  should  conduct  an  analysis  amongst  all 
stakeholders  involved  (client,  future  users,  or  designers),  because  this  increases  the  chance  that  the  program 
will  be  geared  to  everyone’s  needs. 

■ When  designing  ‘normal’  (non-ideological)  educational  products,  designers  should  consider  possible  design 
and  implementation  pitfalls  and  problems  during  the  analysis  phase,  because  it  helps  to  be  prepared  and 
adjustments  can  be  made  if  possible. 


Table  7:  Scores  on  principle  4 


++  + 

- 

? 

Explanation 

TB  1 2 

1 

0 

0 

Design  processes  need  to  be  split  up  into  phases  that 

CD  1 1 

0 

1 

0 

stakeholders  can  recognize.  Each  phase  needs  to  end 

MM  1 2 

1 

0 

0 

formally,  so  that  the  process  can  be  stopped  or  adjusted. 

IT  2 1 

0 

0 

0 

Only  the  next  phase  should  be  planned,  because  that  is 

ET  6 1 

0 

0 

0 

the  one  that  is  most  easy  to  foresee. 

DE  2 1 

0 

0 

0 

13  8 

2 

1 

0 

Arguments  in  favor 

Arguments  against 

Helps  client  to  make  soundly  based  choices;  and 

Not,  when  initial  choices  have  already  been  made. 

satisfied  clients  will  return 

Not  in  case  of  constraining  conditions,  such  as 

+ Alternative  solutions  could  work  better. 

market  factors  or  technical  hindrances. 

+ If  effectiveness  is  more  important  than  efficiency. 

Not  if  specific  solution  type  is  subsidized. 

+ Transfer,  implementation  aspects  and 

Not,  when  designers  need  to  take  specific 

implementation  can  be  studied. 

educational  or  learning  psychology  as  starting  point. 

+ Another  option  is 

to  investigate  applicability  of 

Not  if  designers  wish  to  create  a recognizable 

solutions  that  worked  in  other  situations 

product  to  compete  in  the  market. 

New  design  heuristics 


■ When  the  solution  has  not  been  prescribed  imperatively,  designers  should  carefully  consider  possible 
alternative  solutions  and  choose  one  based  on  relevant  arguments,  because  they  should  always  strive  for  the 
best  solution  and  should  not  entrust  that  that  is  the  one  that  came  first  to  their  minds. 

■ When  designing  learning  systems  for  companies,  designers  should  pay  a lot  of  attention  to  the  analysis, 
because  it  highlights  the  relation  between  problem  definition,  solution  direction,  and  constraints,  which 
facilitates  decision-making. 

■ When  designing  a method  for  schools,  designers  should  choose  a certain  profile  or  concept,  because  that  makes 

them  recognizable  and  identifiable.  This  provides  clarity  (but  is  a producer’s  risk), 
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■ When  designing  a program  for  an  external  client,  designers  should  offer  many  educational  concepts,  because 
the  client  should  be  committed  to  final  products. 

■ Educational  solutions  for  a company  should  be  offered  as  flexibly  as  possible,  because  attunement  to  the 
working  context  is  a determining  factor  for  transfer  and  effectiveness. 

differently,  when  developing  a specific  type  of  product  or  conceiving  a certain  theory  is  a goal,  rather  than  a means. 
In  such  cases,  consideration  of  alternative  solutions  may  only  be  distractive. 

Principle  5:  Designers  Should  Split  The  Design  Process  Into  Phases  With  Formal  Decision  Moments  And 
Concrete  Products,  And  Should  Only  Plan  The  Upcoming  Phase  In  Detail 

75%  Of  the  designers  scored  positively  on  this  principle  (see  Table  8).  Two  out  of  three  IT-designers  partly 
disagreed  with  the  principle.  Designers  from  the  TB,  CD,  and  DE-sector  were  most  positive.  Designers  from  the 
others  sectors  were  most  divided.  While  the  workshop  moderator  read  the  principle,  several  remarks  were  made  by 
designers.  MM10  indicated  that  her  company  works  with  fixed-time,  fixed-money  proposals  towards  clients.  This 
means  that  an  advanced  planning  is  made  for  the  client,  which  should  also  be  kept,  but  that  designers  may  constantly 
change  and  adapt  their  individual  and  internal  plans  during  the  project.  Designer  ET17  argued  as  the  most  important 
that  the  planning  makes  sense  to  stakeholders  and  that  they  recognize  themselves  in  the  planning. 

The  newly  formulated  heuristics  seem  to  refer  to  the  function  and  use  of  planning  in  general,  rather  than  to 
the  specific  assertion  of  the  principle  (split  the  overall  process  into  phases  and  plan  only  the  first  phase  in  detail). 
Both  heuristics  seem  to  focus  on  different  functions  of  planning.  The  first  heuristic  refers  to  the  function  of  a 
planning  document  as  a communication  tool,  and  as  such  supports  a detailed  planning  (in  all  phases).  The  second 
acknowledges  the  complexity  of  many  design  processes,  thus  indicating  that  a planning  can  never  fully  structure  the 
process. 


Table  8:  Scores  on  principle  5 


++ 

+ 

- 

? 

Explanation 

TB 

4 

0 

0 

0 

0 

Design  processes  need  to  be  split  up  into  phases  that 

CD 

2 

1 

0 

0 

0 

stakeholders  can  recognize.  Each  phase  needs  to  end 

MM  2 

0 

1 

1 

0 

formally,  so  that  the  process  can  be  stopped  or  adjusted. 

IT 

1 

0 

2 

0 

0 

Only  the  next  phase  should  be  planned,  because  that  is 

ET 

1 

4 

1 

1 

0 

the  one  that  is  most  easy  to  foresee. 

DE 

2 

1 

0 

0 

0 

12 

6 

4 

2 

0 

Arguments  in  favor 

Arguments  against 

+ 

planning  useful  for  structuring  and  monitoring 

Can  be  a hindrance  when  acceptation  of  the  design 

process 

is  more  important  than  timely  delivery. 

+ 

only  next  phase  can  be  ‘known’ 

+ 

Establishes  clarity  towards  client 

+ 

Note,  phases 

can 

still  be  conducted  partly 

concurrently 

New  design  heuristics 

■ When  designing  educational  products  for  specific  clients,  the  principle  should  be  put  to  work:  designers  should 
present  concrete  work  plans  to  clients  because  these  plans  support  communication  and  establish  clarity. 

■ When  designing  innovative  products  for  large  companies,  the  work  plan  should  be  kept  global,  because  the 

change  process  cannot  be  estimated  in  advance,  and  a detailed  work  plan  could  raise  resistance. 

Principle  6:  For  Efficient  And  Effective  Formative  Evaluations,  Several  (About  Three)  Sources  And  Several 
(About  Three)  Data  Gathering  Instruments  Should  Be  Used 

Designers  were  divided  in  their  scores  on  this  principle  (see  Table  9).  Although  only  one  designer  totally 
disagreed,  half  of  the  group  partly  disagreed  with  the  principle.  The  other  half  was  fairly  or  fully  positive.  This 
division  is  also  evident  in  the  way  the  groups  elaborated  the  arguments  for  the  principle.  Also  within  sectors,  except 
the  IT-sector,  scores  were  heterogeneous.  From  the  arguments  given  in  favor  and  against  the  principle,  it  can  be 
concluded  that  the  factor  time  plays  a decisive  role  in  conducting  or  omitting  on  or  more  formative  evaluation 
activities.  The  fact  that  the  application  of  several  evaluation  methods  may  increase  the  reliability  of  the  conclusions 
is  perceived  as  less  relevant. 
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The  first  new  heuristic  repeats  the  overall  worth  of  evaluation.  In  the  second  heuristic  designers  did  not  so 
much  add  a specific  design  context  but  provide  a constraint  that  exists  in  every  design  situation,  namely  time. 


Table  9:  Scores  on  principle  6 


++ 

+ 

- 

? 

Explanation 

TB 

2 

1 

1 0 

0 

Formative  evaluations  are  often  skipped  because  people 

CD 

1 

0 

2 0 

0 

think  they  are  time-consuming,  expensive,  and  need  a 

MM  0 

2 

1 1 

0 

lot  of  respondents.  The  idea  behind  this  principle  is  that 

IT 

0 

0 

3 0 

0 

evaluations  could  be  conducted  on  a small  scale. 

ET 

2 

1 

4 0 

0 

although  ‘triangulation’  is  needed  to  draw  justifiable 

DE 

0 

2 

0 0 

1 

conclusions. 

5 

6 

11  1 

1 

Arguments  in  favor 

Arguments  against 

+ 

Evaluation  can  be  valuable  and  valid,  and  is 

Formative  evaluation  may  be  excluded  from 

important  for  all  contexts. 

process  if  it  hinders  the  time  range  of  the  project. 

+ 

Formative  evaluation: 

Relying  on  many  resources  may  blur  the  results 

can 

help  prevent  problems  during  process; 

(different  opinions)  and  may  take  too  much  time. 

can 

partly  contribute  to  creating  commitment; 

Usually  one  (client)  or  two  (client  and  content 

can  increase 

product  quality. 

expert)  sources  are  enough. 

+ 

Triangulation  over  respondents  and  data  (sources) 

leads  to  empirically  reliable  conclusions 

New  design  heuristics 

■ In  case  of  educational  design  for  one-off  clients,  designers  should  conduct  formative  evaluations  during  the 
design  and  implementation  phase,  because  formative  evaluation  improves  the  quality  of  the  product,  may 
prevent  problems  further  in  the  design  process,  and  helps  to  create  commitment. 

■ If  they  want  to  design  efficient  and  effective  formative  evaluations  of  a product,  designers  should  not  use  too 

many  sources,  because  it  hinders  the  time  range  of  the  process,  and  because  designers  run  the  risk  that  not  all 
gathered  relevant  information  can  be  processed. 

Principle  7:  In  Order  to  Clarify  Product  Specifications,  Designers  Should  Spend  Their  Time  on  Carefully 
Planned  Formative  Evaluations  of  Early  Versions  of  a Prototype,  Rather  Than  on  an  Elaborate  Preliminary 
Analysis 

Designers  disagreed  highly  on  this  principle,  either  within  or  between  contexts.  Overall,  this  principle  was 
supposed  to  be  different  between  corporate  training  and  education  sectors,  which  again  seem  to  relate  to  generic  and 
site-specific  situations.  Negative  scores  were  dominant  (see  Table  10). 


Table  10:  Scores  on  principle  7 


++  + 

- 

? 

Explanation 

TB  1 0 

1 1 

1 

The  concept  of  formative  evaluation  activities  means  in 

CD  0 1 

2 0 

0 

this  case:  show  intermediate  versions  to  stakeholders 

MM  0 2 

2 0 

0 

with  a specific  goal  and  question.  The  term  preliminary 

IT  0 1 

1 1 

0 

analysis  refers  to  several  analysis  activities  prior  to  the 

ET  0 2 

3 2 

0 

design  phase. 

DE  0 0 

0 2 

1 

1 6 

9 6 

2 

Arguments  in  favor 

Arguments  against 

+ In  the  education  sector,  extensive  analysis  (content- 

Training  needs  and  preferences  can  only  be 

related)  is  often  not  possible.  Thus,  formative 

detected  in  analysis  phase.  Formative  evaluation 

evaluation  is  absolutely  necessary. 

could  help  improve  the  product. 

+ Teachers  prefer  seeing 

a first  product  and  react  to  it. 

above  having  a thorough  analysis  and  wait  for  the 

product. 

New  design  heuristics 

■ In  case  of  design  for  the  context  of  schools  (generic  design!),  designers  should  attribute  major  time  to 

formative  evaluation,  because  they  cannot  rely  on  other  sources  than  market  research  for  changing  the  product. 
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■ When  designing  specific  corporate  training  programs,  designers  should  conduct  a thorough  analysis  because 
that  will  clarify  the  relationship  between  problem  definition  and  direction  for  solution,  and  constraints.  This 
facilitates  decision-making. 

Principle  8:  Designers  Should  Conduct  Formative  Evaluations  Themselves 

Similar  to  the  previous  principle  on  formative  evaluation,  designers  disagreed  on  this  principle.  Scores 
within  sectors  were  also  heterogeneous  (see  Table  1 1).  Although  some  of  the  arguments  refer  to  a specific  type  of 
context  (such  as  politically  sensitive  situations)  the  arguments  mostly  seem  to  refer  to  basic  ideas  about  evaluation 
(the  drive  to  be  objective  versus  the  convenience  of  being  subjective)  or  to  specific  evaluation  qualities  of  the 
individual  designer  or  team. 

The  first  design  heuristic  tries  to  balance  between  objectivity  and  subjectivity.  The  second  heuristic  adds  a 
context  to  the  original  principle. 


Table  11:  Scores  on  principle  8 


++  + 

- 

? 

Explanation 

TB 

2 0 

1 

1 

0 

Formative  evaluation  is  showing  a prototype  or 

CD 

1 1 

0 

1 

0 

intermediate  product  to  clients,  experts,  users,  and  other 

MM  0 2 

2 

0 

0 

stakeholders,  to  get  their  judgments  on  the  product  and 

IT 

0 2 

1 

0 

0 

to  generate  suggestions  for  revision.  If  designers 

ET 

1 2 

3 

1 

0 

conduct  formative  evaluation  themselves,  it  will  save 

DE 

0 1 

1 

0 

1 

time  (efficiency).  Also,  designers  will  get  to  know 

4 8 

8 

3 

1 

where  problems  occur,  so  that  they  can 

prevent/overcome  these  problems  in  next  versions 

adequately  (effectiveness). 

Arguments  in  favor 

Arguments  against 

+ 

Designers  can  evaluate 

more 

purposively  than 

Designers  could  be  biased,  thus  hindering 

someone  who  does  not  know  the  design  process. 

objectivity 

+ 

Designers  can  be  more 

subtle  because  they  know 

Not  in  case  of  politically-sensitive  situations  or 

what  to  probe; 

when  fundamental  choices  need  to  be  made  with 

+ 

Is  efficient,  designers  can  do  it  quickly  themselves 

respect  to  the  content. 

and  can  immediately  apply  the  results; 

Not  when  the  evaluation  gets  too  specific  for 

+ 

Diminishes  ballast. 

designers’  evaluation  expertise. 

New  design  heuristics 

■ If  designs  are  made  in  a team,  the  team  should  evaluate  themselves,  since  they  know  what  to  probe.  The 


evaluator  is  preferably  not  the  designer,  as  this  will  benefit  the  objectivity. 

■ If  formative  evaluations  need  to  be  conducted  on  projects  that  are  politically  insensitive,  or  content-insensitive, 

then  designers  should  conduct  these  themselves,  because  they  are  quick,  have  no  ballast,  can  directly  apply 
results,  and  have  a chance  to  probe. 


Conclusions 

The  study  started  with  the  overall  question  ‘What  are  practical  and  robust  procedural  design  principles, 
related  to  educational  analysis  and  evaluation?’.  To  answer  this  question,  8 design  principles,  primarily  related  to 
analysis  and  evaluation,  were  formulated  based  on  a number  of  reconstructive  studies.  As  such,  these  design 
principles  had  a basis  in  practice.  In  order  to  check  how  robust  these  practical  principles  are,  a workshop  was 
planned  with  24  designers  from  different  design  contexts,  in  which  these  principles  were  discussed  and 
contextualized. 

Overall,  designers  showed  a remarkable  agreement  on  the  majority  of  the  principles.  Although  this  may  be 
due  to  the  rather  general  nature  of  the  principles,  it  also  indicates  that  the  principles  have  potential  for  different 
contexts.  In  the  new  heuristics  that  were  formulated  by  designers,  some  of  the  old  formulations  were  sharpened. 
Some  of  them  were  specified  more  towards  different  contexts  and  thus  increased  insights  into  the  specific  situations 
in  which  the  principles  are  valid.  The  discussion  of  the  principles  shows  that  to  some  extent,  the  specific  design 
situation  influences  the  way  in  which  analysis  and  evaluation  activities  are  conducted.  But  the  discussion  also  shows 
that  the  choice  to  conduct  or  omit  specific  activities,  and  the  perceived  relevance  of  the  activities  rely  on  more  basic 
and  personal  perceptions.  For  example,  some  designers  argue  that  designers  should  never  conduct  formative 
evaluations  themselves  because  they  cannot  be  objective,  while  others  view  this  subjectivity  as  very  positive  for 
conducting  evaluation  themselves,  since  they  are  much  subtle  because  they  exactly  know  what  they  want  to  evaluate 
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and  they  are  more  sensitive  to  the  findings.  Overall,  the  factor  time  shows  to  be  a very  important  and  decisive  factor 
in  the  choice  of  conducting  or  omitting  evaluation  activities.  Although  this  finding  fully  supports  findings  from  other 
reconstructive  studies,  it  partly  contradicts  the  finding  for  principles  1 and  2,  where  designers  indicated  to  support 
the  principle  of  showing  and  discussing  early  prototypes  or  examples  from  other  projects,  which  are  in  fact  kinds  of 
(informal  and  low-profile)  evaluations.  One  possible  explanation  is  that  designers  may  have  different  perceptions  of 
the  term  evaluation,  this  being  rather  formal  and  more  towards  the  end  of  the  process. 

With  respect  to  the  context,  the  amount  of  innovation  of  a situation  or  a product  seems  to  highly  influence 
the  flow  of  the  design  process.  Designers  proposed  an  iterative  approach  in  these  cases,  as  a means  to  tackle  and 
reduce  uncertainty.  The  creation  of  prototypes  early  in  the  design  process  was  regarded  as  useful  because  it  helps  to 
clarify  the  design  specifications  and  their  consequences.  In  this  way,  expectations  can  be  tuned  and  uncertainties 
reduced.  Because  results  are  visible  early  in  the  process,  they  can  be  tested  quickly.  For  these  reasons,  the  use  of 
early  prototypes  was  also  supposed  to  create  commitment.  Because  of  the  amount  of  uncertainty  that  exists  in 
innovative  situations  or  for  innovative  products,  designers  need  not  to  be  too  pretentious  with  their  plans.  In  these 
contexts,  it  seems  to  be  more  efficient  to  keep  the  planning  global  and  to  let  the  process  flow  evolve. 

Several  newly  formulated  heuristics  distinguish  between  the  education  and  the  training  or  corporate 
education  sector  as  a specific  design  context  influencing  the  process.  The  supposed  difference  between  these  sectors 
came  out  especially  in  relation  to  principles  4 and  7.  It  would  be  interesting  to  further  explore  whether  this 
difference  stems  from  differences  in  for  example  target  groups  (kids  versus  adults)  or  function  of  the  design  (aimed 
at  increasing  general  knowledge  versus  specific  skills  or  work  related  knowledge)  or  whether  they  really  refer  to  a 
difference  in  site-specific  situations  in  which  they  can  and  do  know  the  target  group  (which  is  more  often  the  case  in 
organizations)  and  generic  situations  in  which  they  can  not  possibly  know  the  whole  target  group.  Designers 
attached  different  weight  to  analysis  activities  for  both  education  or  training  contexts.  Overall,  the  role  of  analysis  as 
activity  to  steer  the  project  was  regarded  more  important  in  the  context  of  corporate  training  than  in  education, 
because  in  the  former,  a clear  link  can  be  established  between  a problem  and  the  solution.  In  education,  analysis  was 
only  regarded  useful  in  case  of  redesign  of  an  existing  product.  In  case  of  ‘new’  design,  formative  evaluation  was 
supposed  to  have  more  impact.  Also,  the  consideration  of  different  solutions  was  regarded  to  be  more  important  in 
the  corporate  context,  to  help  clients  deliberately  select  a solution  and  get  them  committed  to  ideas.  Only  in  cases 
where  a certain  program  is  prescribed,  considering  alternative  solutions  was  not  regarded  necessary.  For  educational 
publishers,  it  is  important  that  their  products  can  be  recognized.  Therefore,  they  tend  to  develop  a specific  rationale 
of  their  materials  and  do  not  consider  alternatives. 

For  the  validation  of  design  principles,  either  stemming  from  specific  design  research  as  identified  by  Van 
den  Akker,  Gravemeijer,  McKenney  and  Nieveen  (2006)  or  from  reconstructive  design  research,  such  as  the  DATE- 
study,  such  a workshop  can  be  a valuable  activity  in  later  stages  of  the  research  process.  By  bringing  several 
designers  together,  the  researcher  has  quick  and  efficient  access  to  a huge  amount  of  experience  and  contexts,  as 
long  as  careful  selection  of  respondents  has  been  taken  care  of.  The  workshop  seems  to  be  primarily  relevant  for  the 
validation  of  procedural  design  principles  compared  to  substantive  principles.  Whereas  substantive  design  principles 
could  certainly  be  validated  by  evaluating  a product  that  has  been  designed  according  to  the  principles,  the 
validation  of  procedural  principles  by  applying  them  on  a design  process  seems  to  be  less  evident,  given  the  time 
that  is  needed  to  finish  the  process.  In  those  situations,  it  can  be  a good  alternative  to  discuss  the  principles  with 
designers  whose  daily  work  is  to  design. 

While  the  discussion  of  design  principles  in  a workshop  may  help  the  researcher  to  develop  design  theory, 
it  may  also  directly  impact  design  practice,  since  it  provides  designers  with  an  opportunity  to  discuss  and  network 
with  congeners,  also  across  disciplines,  which  they  may  miss  at  their  busy  jobs.  It  helps  them  sharpen  their  own 
ideas  about  what  design  is  or  should  be  and  thus,  directly  impacts  their  work.  As  such,  the  learning  experiences  of 
such  a workshop  feed  both  practitioners  and  researchers. 
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Sharing,  and  Learning 
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University  of  Louisiana  at  Lafayette 

Background  of  the  Project 

Teacher  preparation  programs  throughout  the  United  States  are  being  challenged  to  implement  systemic 
approaches  to  evaluate  the  readiness  of  their  students  to  enter  the  teaching  profession.  Grade  point  average  and 
completion  of  required  courses  are  no  longer  acceptable  evidence  of  degree  program  quality.  Institutions  have 
developed  systematic  assessment  plans  that  detail  data  sources  (i.e.  both  direct  and  indirect  measures  of  student 
learning),  data  analysis  procedures,  and  processes  for  utilizing  these  data  for  program  improvement.  Since  these 
efforts  can  be  time  consuming  and  complex,  many  have  turned  to  technology  as  means  to  support  and  enhance  the 
process. 

This  was  the  case  in  Louisiana,  where,  during  Spring  2001,  leaders  from  21  teacher  preparation  programs 
throughout  Louisiana  began  discussing  the  need  for  a technology-based  system  to  address  these  needs  and  the 
implications  of  the  National  Council  for  the  Accreditation  of  Teaching  Education  (NCATE)  standards  (NCATE, 
2001).  Between  July  2001  and  June  30,  2002,  faculty  and  students  in  the  Division  of  Education  at  Xavier  University 
of  Louisiana  and  the  College  of  Education  at  the  University  of  Louisiana  at  Lafayette  gathered  requirements  and 
implemented  PASS-PORT. 

PASS-PORT,  a web-based  system,  enables  institutions  to  define  key  transition  points,  called  portals  in 
PASS-PORT.  Each  portal  consists  of  a set  of  requirements  students  must  meet  (e.g.  minimum  grade  point  average, 
minimum  rubric  ratings  on  required  artifacts)  providing  a mechanism  for  determining  whether  a student  is  ready  to 
move  into  the  next  phase  of  the  degree  program.  Students  can  create  or  upload  artifacts  that  document  knowledge, 
skill,  and  dispositions.  These  artifacts  can  be  placed  in  electronic  portfolios  and  aligned  to  professional  standards 
and  a reflection  provided  by  students.  Portfolios  can  be  routed  electronically  to  assessment  teams  for  evaluation  of 
students  at  the  portals  (i.e.  key  program  transition  points).  PASS-PORT  has  experienced  success  in  that  20  teacher 
preparation  programs  that  utilized  the  system  for  recent  accreditation  visits  passed  without  stipulations. 

The  Problem 

Though  PASS-PORT  has  helped  institutions  meet  accreditation  requirements,  the  following  problems  have 

emerged. 

Failure  to  follow  best  practice  program  improvement  process.  Though  many  institutions  desire  to  have 
data  for  decision  making  or  to  meet  accreditation  pressures,  they  fall  short  implementing  systematic  assessment 
plans  that  detail  data  sources  (i.e.  both  direct  and  indirect  measures  of  student  learning),  data  analysis  procedures, 
and  a process  for  utilizing  these  data  for  program  improvement.  .For  example,  some  institutions  struggle  to  clearly 
define  degree  program  outcomes  and  select  or  develop  data  sources  to  assess  them.  Many  institutions  collect  data, 
but  fail  to  “loop  back”  and  use  those  data  for  making  decisions  about  improving  the  degree  programs. 

Perceptions  of  extra  responsibilities  and  tasks  imposed  by  PASS-PORT.  PASS-PORT  emphasizes  a top 
down  approach  by  focusing  on  unit  assessment  needs  that  are  driven  by  external  standards  set  by  accreditation 
agencies.  The  emphasis  is  misplaced  on  satisfying  the  accreditation  agency  rather  then  focusing  on  how  each 
individual  in  the  organization  can  improve  program  outcomes.  Many  faculty  have  a negative  attitude  toward  the 
process  because  it  adds  to  their  busy  work  schedule  with  assessment  tasks  they  see  as  having  no  purpose  other  than 
meeting  external  accreditation  pressures. 


Theoretical  Framework 

Literature  related  to  electronic  performance  support  systems  (EPSS)  and  knowledge  management  systems 
(KMS)  provides  a theoretical  framework  to  address  these  problems.  An  EPSS  generally  refers  to  a system  that 
provides  just-in-time  support  for  performance  and  learning  with  a repository  of  information,  resources,  and  tools 
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(Gery,  1991;  Hannafin,  Hill,  & McCarthy,  2000;  Harmon  & Jones,  1999).  PASS-PORT  can  be  perceived  as  an 
EPSS  in  that  PASS-PORT  is  an  electronic  tool  that  facilitates  administrators  and  faculty  in  assessment  data 
gathering  tasks  and  aggregation  tasks.  KMS  is  “a  class  of  information  systems  applied  to  managing  organizational 
knowledge”  (Alavi  & Leidner,  2001,  p.  1 14).  KMS  may  facilitate  one  or  all  of  the  following  processes:  knowledge 
creation,  storage/retrieval,  transfer,  and  application.  PASS-PORT  can  be  categorized  as  a KMS,  because  it  enables 
capturing,  aggregating,  and  retrieving  of  assessment  data. 

Ma  and  Harmon  (2006)  argue  that  EPSS  and  KMS  should  be  integrated  with  learning  technologies  because 
they  have  a shared  theoretical  base;  integrating  the  three  technologies  may  leverage  the  strengths  of  each.  KMS, 
EPSS,  and  learning  technologies  are  all  solutions  to  performance  problems.  KMS  deal  with  managing  knowledge  at 
the  organization  level  to  achieve  strategic  advantage  (Alavi  & Leidner,  2001;  Schwen,  Kalman,  Hara,  & Kisling, 
1998),  whereas  traditional  EPSS  are  more  interested  in  improving  individual  performance  (Laffey,  1995).  Learning 
technologies,  on  the  other  hand,  usually  focus  on  supporting  performance  by  enabling  learning  and  knowledge 
acquisition  at  the  individual  level.  A recent  trend  toward  a holistic  view  of  learning,  knowledge,  and  performance 
provides  a theoretical  base  for  connecting  the  three  areas.  Situated  cognition  (Brown,  Collins,  & Duguid,  1989) 
suggests  that  learning,  knowledge  generation,  and  performance  occur  as  parts  of  the  same  process.  Theories  such  as 
distributed  cognition  (Greenberg  & Dickelman,  2000;  Salomon,  1993)  and  activity  theory  (Engestrom,  Miettinen,  & 
Punamaki-Gitai,  1999)  indicate  that  learning,  knowledge,  and  performance  can  be  enhanced  by  improving  the 
individual  and  the  community,  or  the  tools  and  artifacts  used  by  both.  In  addition,  learning  and  knowledge  at  the 
individual,  group,  and  tool  levels  enhance  each  other. 

Ma  and  Harmon  (2006)  suggest  an  integrated  working/learning  environment  (IWLE)  as  a conceptual  means 
to  integrate  KMS,  EPSS,  and  learning  technologies.  Users  interact  with  the  IWLE  user  interface  while  performing, 
learning  and  managing  their  knowledge.  At  the  user  interface  level,  there  are  three  types  of  tools  available: 
performance  support  tools,  an  intelligent  learning  portal,  as  well  as  community  building  and  knowledge  sharing 
tools.  Performance  support  tools  provide  the  information,  resources  and  devices  needed  for  someone  to  perform 
tasks.  Examples  of  performance  support  tools  include  rule-based  or  case-based  advisory/expert  systems,  workflow 
automation  systems,  decision-support  tools,  data  mining  tools,  productivity  software  such  as  text  processors,  and 
applications  specific  to  a job.  Users  turn  to  the  intelligent  learning  portal  if  their  performance  problems  cannot  be 
resolved  with  the  use  of  performance  support  tools.  The  intelligent  learning  portal  prescribes  customized  learning 
experiences  for  users  based  on  their  backgrounds  and  the  current  problems  they  encounter.  Community  building  and 
knowledge  sharing  tools  can  be  used  together  with  performance  support  tools  and  the  intelligent  learning  portal  to 
facilitate  learning  and  performance  during  social  interactions.  These  tools  support  community  building,  knowledge 
acquisition,  transfer,  storage  and  application.  Examples  of  such  tools  include  groupware,  bulletin  boards,  discussion 
forums,  and  information  searching  tools  that  provide  access  to  various  knowledge  repositories  and  directories.  To 
facilitate  the  integration  of  these  tools,  a knowledge  capture  function  is  embedded  in  all  the  tools  with  which  users 
interact.  A top-down  process  captures  data  from  the  interface  and  sends  it  down  to  the  bottom  objects  and  data 
layers.  A bottom-up  process  gathers  reusable  information  from  the  data  and  object  layer,  repurpose  it  for  specific 
presentation  formats  and  sending  it  to  the  users  in  the  formats  such  as  online  training,  job  aids,  simulations,  and  etc. 

In  summary,  the  following  guidelines  can  be  derived  from  the  literature  reviewed  above:  1)  provide  tools 
and  resources  to  support  best  practice  work  process;  2)  capture  all  data  entered  by  the  user  in  the  process;  3) 
repurpose  the  data  to  support  user  learning  and  performance  as  well  as  to  facilitate  knowledge  sharing. 

Problem  Analysis  and  Solution  Development 

The  theoretical  framework  described  above  provides  a lens  for  us  to  analyze  the  problems  we  have 
encountered  in  implementing  PASS-PORT  and  to  help  us  generate  solutions.  The  first  problem  is  the  failure  of  some 
organizations  to  follow  best  practice  program  improvement  process.  One  of  the  attributes  of  EPSS  is  that  the  work 
process,  business  strategies,  and  best  practices  should  be  built  into  the  system  to  ensure  their  implementation.  PASS- 
PORT lacks  direct  support  for  the  entire  program  improvement  process.  Data  collection  at  transition  points  as  well 
as  data  aggregation  and  retrieval  are  supported.  However,  PASS-PORT  does  not  support  the  other  steps  in  the 
program  improvement  process,  including  assessment  plan  definition,  data  interpretation,  noting  actions  taken,  and 
results  dissemination.  The  lack  of  this  support  has  resulted  in  institutions  beginning  data  collection  without  making 
thoughtful  decisions  on  the  overall  assessment  plan  (i.e.  identifying  key  outcomes  to  measure  and  determining  data 
sources  to  be  used  as  evidence).  In  addition,  data  interpretation  may  or  may  not  happen  on  a regular  basis  and  action 
may  not  be  taken  to  address  issues  identified  in  the  assessment  data.  Directly  supporting  the  whole  program 
improvement  process  may  enforce  the  implementation  of  best  practice.  Therefore,  we  decided  to  add  functionality  in 
PASS-PORT  to  facilitate  the  entire  process. 
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The  second  problem  is  faculty  perceptions  of  extra  responsibilities  and  tasks  imposed  by  PASS-PORT. 
Faculty  often  view  the  implementation  of  assessment  systems  as  a burden  imposed  to  only  meet  external 
accreditation  pressures.  This  problem  is  similar  to  what  some  KMS  developers  have  encountered.  For  example,  In  a 
KMS  project,  the  users  almost  unanimously  stated  that  they  would  not  use  this  KMS  in  their  daily  operation, 
because  it  was  not  integrated  into  their  work  environment  (Johnson,  Birnbaum,  Bareissand,  & Hinrichs,  2000). 

Using  the  KMS  would  take  them  out  of  their  work  context  and  add  to  their  workload.  To  address  problems  like  this, 
they  added  an  EPSS  interface  to  the  KMS.  In  other  words,  they  built  data  capturing  and  data  retrieval  into 
employees’  daily  work  process  which  is  supported  by  an  EPSS.  Along  the  same  line  of  thinking,  in  our  case, 
embedding  the  assessment  data  gathering  process  into  faculty  members’  daily  work  may  decrease  the  workload  and 
lead  to  positive  perceptions  of  the  tool. 

How  do  we  achieve  this  goal?  PASS-PORT  was  originally  designed  for  evaluating  students  at  transition 
points.  Students  upload  artifacts  that  document  knowledge,  skills,  and  dispositions,  place  these  artifacts  in  portfolios 
that  are  submitted  in  order  to  gain  permission  to  move  through  a transition  point.  Faculty  evaluate  the  portfolios  to 
judge  whether  the  student  is  ready  to  move  into  the  next  phase  of  the  degree  program.  This  process  occurs  outside  of 
the  context  of  any  course,  so  faculty  members  perceive  it  as  “extra  work”  imposed  by  the  assessment  system.  To 
address  this  problem,  we  consulted  other  assessment  process  and  decided  to  choose  one  that  uses  course-based 
assessment  to  drive  program  and  college  level  assessment  (Walvoord,  2004).  Course-based  assessment  is  an 
established  process  familiar  to  students  and  faculty,  so  it  is  ideal  to  leverage  it  in  supporting  both  program  and 
college  assessment.  A misstep  in  the  original  design  of  PASS-PORT  was  a failure  to  realize  the  power  that  course- 
based  assessment  can  bring  to  program  and  unit  assessment.  Faculty  members  are  already  engaged  in  a process  of 
continual  course  improvement  where  they:  define  course  goals  and  objectives,  measure  the  degree  to  which  these  are 
met,  and  then  utilize  this  information  to  make  improvements  to  the  course.  The  process  can  be  leveraged  not  only  to 
inform  the  faculty  member  of  course  effectiveness,  but  also  to  provide  vital  data  for  program  and  unit  assessment. 
Many  higher  education  institutes  have  adopted  course  management  systems  (CMS)  to  deliver  courses  and  facilitate 
course-based  assessment.  To  use  course-based  assessment  to  drive  program  and  unit  assessment,  we  need  to  develop 
core  CMS  functionality  and  integrate  the  CMS  with  PASS-PORT. 

The  concept  of  an  IWLE  has  inspired  us  to  evaluate  existing  CMS  to  identify  the  gap.  Traditional  CMS 
support  course  design,  delivery,  management,  and  assessment  tasks.  However,  it  fails  to  provide  pedagogical 
support  for  completing  these  tasks.  Ideally,  an  EPSS  should  support  all  aspects  of  the  major  tasks  in  a work  process. 
A preliminary  study  indicates  that  an  EPSS  is  positively  perceived  by  faculty  as  a tool  to  provide  just-in-time 
support  needed  to  design  and  deliver  courses  (Ma,  2005).  Therefore,  we  decided  to  add  functionality  to  support  the 
implementation  of  appropriate  pedagogy.  Details  of  the  functionality  are  presented  in  the  next  section. 

Moreover,  traditional  CMS  does  not  support  the  course  improvement  process.  An  IWLE  type  of  system 
may  provide  opportunity  to  support  this  process.  Knowledge  capturing  and  aggregation  tools  may  collect  course 
assessment  data,  teacher  notes  and  reflection,  as  well  as  data  indicating  how  the  course  is  meeting  program 
outcomes.  These  data  is  presented  to  faculty  through  an  EPSS  interface  to  encourage  faculty  to  reflect  on  the 
effectiveness  of  the  course.  Additionally,  the  system  captures  best  practice  activities,  sample  rubrics,  and  issue 
solutions.  These  data  can  be  stored  and  provided  to  faculty  to  help  them  improve  courses.  Learning  technologies  can 
also  play  an  important  role.  If  the  on-demand  support  is  inadequate  to  meet  faculty  needs,  learning  technologies  such 
as  a faculty  development  portal  give  them  opportunities  to  go  through  tutorials,  workshops  to  receive  informal  and 
formal  training.  Best  practices  in  teaching  can  be  captured  and  repurposed  for  use  in  the  faculty  development  portal. 

Ideally,  an  IWLE  should  support  all  the  work  processes  of  users.  Previous  research  shows  that  faculty 
members  need  a “one-stop  shop”  of  resources  to  help  them  with  teaching  (Ma,  2005).  Following  the  same  logic,  one 
would  expect  that  they  would  benefit  if  a “one-stop  shop”  is  provided  to  support  all  their  work  processes,  including 
teaching,  assessment,  advising,  community  work,  scholarship,  and  etc.  Along  this  line  of  thinking,  we  added 
functionality  to  support  advising  and  merit  and  promotion  processes  because  they  are  easier  to  support  with  a 
system.  Future  effort  may  be  needed  to  provide  support  to  other  processes. 

Falcon:  An  Integrated  System  to  Support  Performance,  Assessment, 

Knowledge  Sharing,  and  Learning 

The  ultimate  goals  of  establishing  an  assessment  process  is  not  only  to  meet  accountability  expectations 
from  accreditation  agencies,  but  more  importantly  to  develop  a mechanism  to  enable  ongoing  assessment  and 
improvement  at  the  course,  program,  and  college  levels.  We  argue  that  to  have  users  buy  into  the  program 
improvement  process,  assessment  procedures  should  be  an  integral  part  of  the  daily  work  process,  and  the  data 
gathered  should  be  used  to  facilitate  learning  and  improvement.  Based  on  this  assumption,  we  have  created  Falcon 
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as  an  integrated  system  to  support  performance,  assessment,  knowledge  sharing,  and  learning  at  course,  program, 
and  college  levels.  This  section  describes  the  functionality  of  Falcon. 

Program  Improvement 

Falcon  provides  sophisticated  tools  and  mechanisms  for  supporting  organizations  in  program  improvement. 
Key  components  include:  degree  program  transition  point  definition,  assessment  plan  configuration,  course-based 
assessment  drives  program  assessment,  explicit  linkage  of  outcomes  at  all  levels  of  an  organization,  and  support  for 
external  performance  data. 

Degree  program  transition  points.  Falcon  supports  the  definition  of  degree  program  transition  points  (i.e. 
portal),  articulation  of  requirements  (including  artifacts  to  be  evaluated),  creation  of  evaluation  instalments  (i.e. 
rubrics)  to  evaluate  artifacts  and  performances  at  each  transition  point,  and  setting  the  level  of  performance  required 
to  move  through  a transition  point.  Candidates  submit  artifacts  to  meet  portal  requirements;  faculty  view  artifacts 
and  the  evaluation  rubric  side-by-side  to  perform  evaluation.  The  transition  point  component  serves  (1)  as  a gate 
were  the  readiness  of  the  candidate  to  move  into  the  next  phase  of  the  program  is  judged  and  (2)  as  a data  source  for 
drawing  conclusions  on  the  quality  of  degree  programs. 

Assessment  plans.  Falcon  supports  the  entire  program  improvement  process,  including  defining  assessment 
plans,  collecting  data,  aggregating  data,  interpreting  data,  taking  action,  and  disseminating  results  to  stakeholders. 
Assessment  plans  are  built  upon  degree  program  outcomes.  For  each  program  outcome,  any  number  of  data  sources 
can  be  selected  as  potential  sources  of  evidence  that  the  outcome  is  being  met.  Data  sources  include:  user 
demographics,  surveys,  artifact  evaluations,  test  scores  (e.g.  Praxis),  and  external  data.  A rationale  is  provided 
indicating  how  the  data  source  will  be  used  to  judge  whether  the  outcome  is  met.  Once  the  data  sources  are  selected, 
Falcon  automatically  aggregates  the  data  sources.  Falcon  also  supports  regular  interpretation  and  executive  review. 
On  a regular  basis,  the  data  sources  can  be  reviewed,  conclusions  drawn,  weakness  articulated,  and  summary  posted 
to  the  assessment  plan.  Additionally,  after  the  executive  review  committee  reviews  the  data  and  interpretation 
summary,  any  actions  taken  can  be  posted  to  the  assessment  plan.  Falcon  allows  all  faculty  to  review  assessment 
plans  including  all  program  outcomes,  associated  data  sources  and  rationale,  aggregated  data,  interpretation 
summaries,  weakness  identified,  and  actions  taken.  These  features  ensure  that  the  model  process  is  followed  and 
organizations  actually  “loop-back”  to  improve  programs  based  on  data. 

Course-based  assessment  drives  program  and  college  assessment.  Course  assignments  rated  by  the 
instructor  alone  may  not  be  adequate  to  support  program  and  college  assessment.  Walvoord  (2004)  suggests  that 
adding  other  raters  may  render  course  assignment  a legitimate  tool  for  program  and  college  assessment.  In  Falcon, 
tools  are  provided  to  automatically  send  a percentage  of  course  assignments  to  outside  raters  to  meet  program 
assessment  requirements.  This  functionality  helps  save  limited  resources.  In  addition,  faculty  members  do  not  feel 
they  have  to  do  “extra”  tasks  to  complete  program  and  college  level  assessment.  Such  positive  perceptions  will  help 
ensure  program  improvement  procedures  are  carried  out. 

Explicit  linkage  of  outcomes  at  all  levels  of  an  organization.  Falcon  supports  accreditation  efforts  at  all 
levels  of  an  organization  (i.e.  University,  College,  Program).  Though  faculty  are  familiar  with  course-based 
assessment  processes,  they  are  often  unfamiliar  with  the  connection  between  course  outcomes  and  program  / college 
outcomes.  To  address  this  issue,  Falcon  explicitly  represents  the  linkage  between  course  outcomes,  program 
outcomes,  as  well  as  unit  outcomes  and  provides  tools  for  faculty  to  view  the  impact  of  their  teaching  on  program 
and  unit  outcomes.  For  example,  consider  the  situation  where  two  different  undergraduate  courses  have  primary 
responsibility  for  a program  outcome.  A tool  that  allows  a faculty  member  to  aggregate  data  by  program  outcome 
allows  him/her  to  see  the  degree  to  which  students  are  meeting  program  expectations.  Next,  she  might  “drill  down” 
into  the  program  outcome  and  see  a breakdown  of  the  data  by  the  courses  that  are  linked  to  it.  She  then  might  view 
the  performance  of  students  that  have  come  through  her  course.  Falcon  is  designed  to  explicitly  represent  the 
linkage  between  course,  program,  and  college  outcomes. 

Support  for  external  performance  data.  Falcon  accommodates  external  sources  of  data  including  tutorials, 
rich  learning  environments  (e.g.  problem-based  learning  environments),  and  immersive  video  games.  Increasingly 
faculty  are  recognizing  the  importance  of  electronic  learning  environments  as  a means  to  provide  opportunities  for 
students  to  demonstrate  authentic  performance  albeit  virtual  settings.  Once  configured,  external  learning 
environments  can  transparently  persist  performance  data  to  Falcon.  These  data  can  then  be  included  in  course, 
program,  or  college  assessment  processes. 
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Course  Design,  Delivery,  and  Improvement 

Falcon  provides  common  CMS  tools  to  support  course  delivery,  communication,  and  collaboration.  For 
example,  tools  are  available  for  faculty  to  disseminate  course  content,  including  documents,  text,  and  graphics,  and 
rich  interactive  content.  There  are  also  tools  that  facilitate  dialog  and  collaboration,  including:  alerts,  threaded 
discussion  forums,  evaluation  of  postings  using  rubrics,  email,  chat,  peer  evaluation  and  feedback,  and  file  sharing. 

Moreover,  Falcon  has  the  following  unique  characteristics  that  support  course  design,  delivery,  and 
improvement:  course  templates  to  ensure  consistent  implementation  of  courses,  just-in-time  support  for  course 
design,  delivery,  and  improvement,  portal  to  both  technical  and  pedagogical  resources  and  tools,  and  sharing  of  best 
practices  and  teaching  experiences. 

Course  templates  to  ensure  consistent  implementation  of  courses  in  the  program.  Course  characteristics  are 
inherited  from  department  definition.  A department  can  define  a course  template,  including  course  details  such  as: 
course  outcomes  (aligned  with  program  outcomes),  required  projects  and  associated  scoring  instruments.  Course- 
based  rubrics  can  be  aligned  with  course  outcomes,  program  outcomes,  unit  outcomes,  and  professional  standards. 
Additionally,  course-based  rubrics  can  inherit  attributes  from  unit  approved  instruments,  departmental  approved 
rubrics,  and  course  approved  rubrics.  When  a course  is  created  by  a faculty  member,  it  automatically  includes 
required  elements. 

Just-in-time  support  for  course  design,  delivery,  and  improvement.  Research  shows  that  importance  of  on- 
demand  support.  Faculty  members  expressed  the  need  to  acquire  relevant  knowledge  that  addresses  their  specific 
concerns  about  teaching  (Laga  & Elen,  2001)  and  that  can  be  applied  immediately  (Carbonaro,  Snart,  & Goodale, 
2002).  They  are  especially  interested  in  learning  from  concrete  examples  provided  by  experienced  peers  (Carbonaro 
et  al.,  2002;  Laga  & Elen,  2001).  Falcon  has  the  potential  to  meet  this  need.  It  provides  on-demand  examples  of 
activities,  assessments,  and  best  practice  solutions  to  problems.  For  example,  when  a faculty  member  is  designing  a 
discussion  forum  activity  in  Falcon,  the  system  provides  example  discussion  activities,  templates  for  activity 
instruction,  and  the  associated  rubrics  faculty  may  reuse  to  create  their  own.  If  a faculty  member  runs  into  an  issue, 
for  example,  students  are  not  posting  meaningful  messages  on  the  discussion  forum,  s/he  may  search  the  issues 
library  to  find  out  how  other  instructors  have  addressed  similar  problems.  During  and  after  teaching,  Falcon  helps 
the  professors  monitor  their  teaching  with  a set  of  tools,  including  course  summary  data,  notes,  issues  log, 
connection  to  program  data,  connections  to  other  courses.  If  course  improvement  is  needed,  Falcon  provides  best 
practice  examples  to  help  faculty  redesign  the  course. 

Portal  to  both  technical  and  pedagogical  resources  and  tools.  In  a previous  study  investigating  faculty 
needs  (Ma,  2005),  faculty  members  asked  for  a “one-stop  shop”  of  resources  to  help  them  with  teaching.  They 
wanted  a gateway  to  the  following  types  of  knowledge  organized  around  their  teaching:  technical  knowledge, 
content  knowledge,  pedagogical  and  content  pedagogical  knowledge,  as  well  as  experiential  knowledge.  A professor 
in  this  study  commented  the  importance  of  having  a portal  to  both  pedagogical  and  technical.  She  stated  that  WebCT 
support  staff  usually  provide  technical  help  on  how  to  do  something  in  CMS,  but  do  not  help  with  pedagogy.  And 
the  center  for  teaching  effectiveness  will  tell  you,  pedagogically,  how  to  do  activities  but  not  provide  information 
about  the  technical  aspect.  Falcon  provides  a portal  to  both  types  of  tools  and  resources.  For  example,  when  an 
instructor  creates  a discussion  board  activity  in  the  Assignment  Tool,  Falcon  not  only  walks  the  instructor  through 
the  steps  to  actually  create  the  forum,  but  also  provide  pedagogical  support  through  examples,  advices,  and 
templates. 

Sharing  of  best  practices  and  teaching  experiences  in  the  community.  Lee  Shulman,  a leader  and  long-term 
advocate  of  teaching  improvement,  called  to  “make  teaching  community  property”  (Shulman  & Hutchings,  2004). 
His  recent  model  of  teaching  improvement  includes  the  community  as  a key  component  that  facilitates  teacher 
learning  (Shulman  & Shulman,  2004).  Systems  have  been  developed  to  capture  and  share  teaching  examples 
(Kolodner,  1991;  Krueger,  Boboc,  & Cornish,  2003;  Wang,  Means,  & Wedman,  2003).  However,  content  collection 
is  a very  time-consuming  task  in  developing  these  systems.  In  a study  that  examines  faculty  perceptions  of  an  online 
teaching  case  library,  faculty  expressed  the  desire  to  share  their  work,  but  they  commented  that  they  may  not  have 
time  to  do  so  (Ma,  2005).  Falcon  may  help  address  the  problem.  The  Assignment  tool  not  only  scaffolds  faculty  on 
how  to  develop  an  activity,  it  also  captures  the  activities  and  the  scoring  instruments  developed  by  them.  The  Issues 
Library  allows  faculty  to  search  for  solutions  and  to  store  their  own  issues  and  solutions  for  future  reference.  With 
these  tools,  faculty  may  contribute  their  own  to  the  community  without  extra  effort.  They  only  need  to  click  on  a 
checkbox  to  make  the  activities  and  issues  they  created  available  to  the  community. 
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Advising 


Advising  students  during  their  academic  career  is  a key  factor  in  improving  retention  rates.  In  many 
institutions,  this  responsibility  falls  to  the  faculty  member.  Falcon  was  designed  with  the  following  functionality  to 
make  the  advising  process  more  efficient  and  effective:  monitoring  of  degree  progress,  advising  appointments, 
advising  records,  and  advising  frequently-asked-questions. 

Monitoring  of  degree  progress.  Each  degree  program  defined  in  Falcon  consists  of  a collection  of  required 
courses  and  transition  points.  As  a student  receives  final  course  grades,  it  is  displayed  next  to  the  required  course. 

As  transition  point  requirements  are  completed,  an  indication  of  whether  the  student  has  met  the  minimal 
requirements  for  that  requirement  is  displayed.  This  enables  the  student  and  faculty  advisor  to  remain  current  on  the 
students  progress  toward  completing  degree  requirements. 

Advising  appointments.  Advisors  are  able  to  define  advising  appointment  slots  in  which  students  can 
select.  This  simplifies  the  process  in  communicating  to  students  available  appointments  and  supporting  students  in 
selecting  one  that  fits  their  schedule. 

Advising  record.  As  students  select  an  advising  appointment,  they  are  provided  an  opportunity  to  select 
courses  they  wish  to  take  in  the  upcoming  semester.  The  student  is  able  to  view  the  required  degree  curriculum  and 
grades  for  those  courses  they  have  already  completed.  Once  submitted  by  the  student,  the  faculty  member  is  able  to 
view,  edit,  and  approve  the  advising  record. 

Faculty  Development 

In  Scholarship  Reconsidered:  Priorities  of  the  Professoriate  (1990),  Ernest  Boyer  makes  a compelling  case 
for  America’s  colleges  and  universities  to  expand  our  view  of  scholarship  and  how  it  is  assessed.  Falcon  features 
can  help  support  the  demonstration  and  assessment  of  scholarship.  Boyer’s  expanded  view  of  scholarship  (i.e. 
scholarship  of  discovery,  integration,  application,  and  teaching)  provides  a useful  framework  for  examining  Falcon’s 
features. 

Portfolio  approach  for  merit,  tenure,  promotion.  Falcon  supports  the  creation  of  reflective  portfolios. 
Portfolios  are  ideal  for  documenting  faculty  development.  Faculty  can  create  or  upload  artifacts  that  document 
professional  growth.  These  artifacts  can  be  placed  in  electronic  portfolios  and  aligned  to  professional  standards  and 
a reflection  provided.  Portfolios  can  be  routed  electronically  to  assessment  teams  for  evaluation. 

Artifact  templates  for  consistent  documentation.  Artifact  templates  can  be  developed  to  ensure  consistent 
data  collection.  For  example,  a template  can  be  developed  to  document  grant  Rinding  received.  Template  fields 
might  include  Rinding  agency,  scope  of  the  grant  program  (e.g.  local,  state,  and  national),  total  dollar  amount, 
number  of  students  funded  by  grant.  The  artifact  template  insures  that  data  collected  is  consistent.  As  faculty 
receive  grants,  they  simply  document  the  information  in  Falcon.  This  artifact  is  then  available  for  use  in  faculty 
evaluation  and  institutional  reporting.  The  artifact  templates  also  provide  a way  to  make  organizational  priorities 
explicit.  For  example,  if  an  institution  wants  to  encourage  faculty  to  engage  students  in  service  learning,  an  artifact 
template  can  be  created  that  provides  for  a mechanism  to  collect  key  data  on  service  learning  projects.  These  data 
can  then  be  considered  in  annual  faculty  review  and  is  available  for  aggregation  at  the  departmental,  college,  or 
university  level. 

Teaching  artifacts.  To  facilitate  documentation  of  teaching,  Falcon  creates  a summary  artifact  for  course 
taught.  The  summary  report  includes  data  such  as:  course  outcomes  (linked  to  program  outcomes),  student 
demographics,  student  performance  on  course  assignments,  complete  assignment  requirements  and  scoring 
instillments,  and  comparison  of  student  performance  across  sections  and  semesters. 

Service  artifacts.  Falcon  is  designed  to  foster  a community  of  practitioners.  Contributions  to  the 
community  are  valuable  service.  Falcon  supports  documentation  of  contributions  to  this  community  through  shared 
activities  and  assessments.  Additionally,  advising  activity  is  document  by  Falcon  providing  another  means  for 
demonstrating  service  to  the  organization. 
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Introduction 

Providing  opportunities  for  students  to  gain  experience  in  applying  what  they  have  learned  in  everyday 
authentic  situations  is  a challenge  facing  instructional  designers  and  teachers.  Schools  typically  emphasize  the 
transfer  of  abstract  concepts  where  knowledge  is  viewed  as  a discrete  object  residing  outside  of  the  individual  and 
learning  occurs  out  of  context  (Brown,  Collins,  & Duguid,  1989). 

Problem-based  learning  (PBL)  has  been  widely  advocated  as  an  instructional  approach  that  provides 
opportunities  for  engagement  in  authentic  contexts  while  completing  authentic  tasks  (i.e.  problem  solving).  A rich 
body  of  research  conducted  with  adult  learners  suggests  that  PBL  is  particularly  effective  in  helping  learners  to 
develop  deep  understanding  of  principles  within  a domain  (Gijbels,  Dochy,  Bossche,  & Segers,  2005).  However, 
the  literature  on  PBL  points  to  problems  that  may  make  its  use  with  young  learners  difficult.  Many  students  have 
difficulty  with  the  student  directed  learning  required  in  PBL  (Hoffman,  Ritchie,  1997).  The  challenges  of 
implementing  PBL  in  a classroom  require  scaffolding  (Koschmann,  Kelson,  Feltovich,  & Barrows,  1996). 

Virtual  learning  environments  that  utilize  a problem-based  approach  provide  opportunities  for  creating 
authentic  contexts,  engaging  students  in  authentic  tasks,  and  providing  scaffolding  to  support  learning.  One  type  of 
scaffolding  that  is  often  employed  is  the  use  of  cases. 

This  paper  explores  the  potential  of  having  learners  explicitly  compare  two  cases,  a process  called 
analogical  encoding,  while  working  in  a problem-based  virtual  learning  environment.  Research  on  analogical 
encoding  suggests  that  teaching  students  to  compare  cases  side-by-side  improves  knowledge  acquisition  and 
transfer.  Analogical  encoding  aids  students  in  attending  to  the  deep  underlying  structures  (i.e.  principles)  of  cases 
thereby  reducing  potential  distraction  of  surface  features.  This  paper  intends  to  translate  research  findings  on 
analogical  encoding  to  a scaffolding  strategy  in  a problem-based  virtual  learning  environment. 

Analogical  Reasoning 

Analogical  reasoning  is  widely  accepted  to  be  important  in  learning  and  understanding.  Novel  problems 
can  be  solved  by  reference  to  previously  understood  situations  (Kurtz,  Mao,  and  Gentner  2001). 

In  its  simplest  form,  analogical  learning  results  from  noticing  similarity  between  a well-understood  “base” 
domain  and  a less  well  understood  “target”  (or  “test”)  domain.  Solving  a problem  in  the  target  domain  can  be 
facilitated  by  mapping  corresponding  aspects  of  the  target  domain  to  the  base  domain,  and  adapting  a solution  from 
the  base  domain. 

A number  of  learning  environments  utilize  cases  to  scaffold  learning.  For  example,  Roger  Schank  utilized 
“war  stories”  from  domain  experts  in  providing  just-in-time  support  to  students  engaged  in  goal-based  scenarios 
(Schank,  Fano,  Jona,  & Bell,  1993).  Alien  Rescue  provides  video-based  vignettes  to  support  middle  school  students 
in  designing  probes  to  be  sent  to  worlds  in  our  solar  system  (Liu,  Williams,  & Pedersen,  2002).  Cognitive 
flexibility  hypertexts  utilize  “mini-cases”  that  facilitate  the  learner  in  “cross-crossing”  the  conceptual  landscape 
(Spiro  & Jehng,  1990).  In  Learning  by  Design  classrooms  (Kolodner,  Owensby,  & Guzdial,  2003),  design  cases 
are  provided  for  middle  school  students  to  develop  scientific  concepts  and  generate  solutions  to  design  problems. 

However,  the  actual  process  of  retrieving  and  applying  relevant  past  knowledge  (the  “transfer”  problem)  is 
not  trivial.  It  has  been  variously  shown  that  even  when  appropriate  bases  are  stored  in  memory,  retrieval  often  fails 
(Perfetto,  Branford,  & Frands,  193;  Ross,  1989;  Weisberg,  DiCamillo,  & Phillips,  1978,  all  cited  in  Kurtz  et  al. 
2001).  In  addition,  while  structural  similarity  is  the  main  predictor  of  suitability  for  problem  solving,  surface 
similarity  has  been  shown  to  be  a better  predictor  of  retrieval.  Furthermore,  retrieval  is  only  part  of  the  problem: 
once  the  base  is  retrieved,  the  relevant  solution  must  be  adapted  to  the  target  problem  (Novick  and  Holyoak,  1991, 
andNovick,  1995). 
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Though  the  idea  of  providing  cases  to  support  learning  is  appealing,  individuals  are  often  unable  to  retrieve 
and  apply  cases  to  current  problems  (Gentner,  Loewenstein,  Thompson,  2004).  Gentner  et.  al.  argue  that  one  factor 
influencing  this  poor  performance  is  that  the  cases  are  likely  encoded  in  situation  specific  format.  The  solution 
proposed  and  researched  by  Gentner’ s research  team  is  having  students  compare  cases  side-by-side  resulting  in 
improvements  in  retention  and  transfer. 


Analogical  Encoding 

Analogical  encoding  (also  known  as  mutual  alignment)  is  a mechanism  for  making  common  structure  more 
salient  by  comparing  examples  (Gentner  and  Markman,  1997).  It  is  based  on  the  theoretical  framework  of  Gentner’s 
(1983)  structure  mapping  theory,  which  posits  that  analogy  involves  the  identification  of  correspondences  between 
the  structures  corresponding  to  two  domains  and  the  transfer  of  information  from  the  more -understood,  or  base 
domain,  to  the  less-understood,  or  target  domain.  In  analogical  encoding,  the  two  domains  are  equally  novel  to  the 
learner,  thus  each  acts  as  both  base  and  target  with  respect  to  the  other. 

Analogical  encoding  promotes  learning  in  two  ways.  First,  because  the  learner  transfers  information 
between  the  two  situations,  the  representation  of  each  of  the  situations  becomes  enriched  with  information  more 
evident  in  the  other  situation.  Second,  the  identification  of  correspondences  between  structured  representations  of 
each  the  two  situations  allows  the  learner  to  strip  away  details  that  are  not  relevant  to  the  common  structure,  leaving 
behind  an  abstract  representation  of  the  structures  that  can  then  be  applied  to  novel  analogous  situations,  regardless 
of  whether  or  not  the  novel  situations  share  surface  similarities  with  the  originally  learned  situations. 

There  is  some  existing  evidence  that  comparison  can  facilitate  learning.  Gick  Holyoak  (1983)  found  that 
instructing  participants  to  describe  the  similarities  between  two  analogs  led  to  formation  of  abstract  schemas  and 
facilitated  transfer  to  a further  problem.  Catrambone  and  Holyoak  (1989)  showed  that  inducing  participants  to 
compare  multiple  examples  improved  performance  on  long-term  transfer  across  contexts.  Cummins  (1992)  found 
that  when  learners  were  given  questions  that  required  comparison  of  analogous  problems,  they  developed  deeper 
understanding  of  the  solution  principles  than  did  learners  who  received  an  equal  number  of  noncomparison 
questions.  Schwartz  and  Bransford  (1998)  conducted  classroom  studies  to  show  that  inducing  learners  to  generate 
distinctions  between  contrasting  cases  led  to  the  development  of  more  differentiated  knowledge  structures. 
Loewenstein,  Thompson,  and  Gentner  (1999)  found  that  graduate  management  students  who  compared  two 
analogical  cases  were  nearly  three  times  more  likely  to  incorporate  the  common  strategy  into  a subsequent 
negotiation  task  than  were  students  given  the  same  cases  separately.  These  and  related  findings  suggest  that 
analogical  encoding  can  invite  abstraction  and  promote  transfer  (see  also  Gentner  & Namy,  1999). 

In  summary,  research  on  analogical  encoding  (i.e.  learning  by  comparing  two  cases)  suggests  that  teaching 
students  to  compare  cases  side-by-side  improves  knowledge  acquisition  and  transfer.  Analogical  encoding  aids 
students  in  attending  to  the  deep  underlying  structures  (i.e.  principles)  of  cases  thereby  reducing  potential  distraction 
of  surface  features. 


Overview  of  Learning  Environment 

Our  team  is  currently  developing  a four-week  7th  grade  life  science  curriculum  prototype  that  will  consist  of 
approximately  10  class  periods  of  hands-on  classroom-based  activities  and  approximately  10  hours  of  video  game 
activities.  The  game,  to  be  built  using  the  Torque  Game  Engine  (http://garagegames.com),  features  intriguing 
storyline,  immersive  3-D  representation  of  context  and  quests  (problems),  simulations,  tools  and  resources  that 
support  quest  completion,  choice  of  roles  and  tasks,  levels  of  play,  record  keeping,  as  well  as  real  time  interaction 
and  feedback.  Quests  in  the  virtual  environment  are  supplemented  by  classroom-based  activities,  which  address 
students’  knowledge  gaps  revealed  during  game  play  and  extend  problem  solving  from  the  virtual  and  fictitious 
world  to  similar  problems  in  the  local  community. 

Styled  after  highly  successful  commercial  products  like  World  ofWarcraft  and  Everquest,  the  role-playing 
game  has  science  fiction/fantasy  setting.  Pursuing  larger  strategic  objectives,  the  player  character  will  be  challenged 
with  a variety  of  “quests;”  that  is,  player-characters  must  pose  questions  and  seek  answers,  investigate  mysteries, 
formulate  hypotheses,  gather  evidence  and  information,  use  appropriate  tools  and  techniques,  and  ultimately  take 
action  to  solve  problems  presented  within  quests.  These  quests  form  the  main  plotlines  of  the  interactive  narrative 
and  will  provide  the  immediate  motivations  for  player-character  activities  in  the  game.  Each  quest  is  designed  to 
achieve  specific  learning  goals. 

The  game  takes  place  amid  an  ancient  conflict  between  two  sentient  species  and  their  struggle  for 
dominance  on  a planet  in  another  solar  system.  It  is  a warm,  wet  world  of  stormy  oceans,  dotted  with  countless 
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islands  and  a single  small  continental  landmass.  While  not  technologically  sophisticated,  the  planet’s  two  rival 
sentient  species  have  reached  a turning  point  in  their  evolutionary  history  where  it  is  likely  that  one — the  Mruk- 
ma — will  likely  drive  the  other — the  Sheft-ma — into  extinction. 

The  Mruk-ma  are  an  aggressive,  sea-faring  species,  while  the  Sheft-ma  are  city-builders  who  make  their 
home  in  “The  Coastlands,”  along  the  marshy  seashores  and  river  valleys  of  Mertis’  lone  continent.  For  the 
vulnerable  Sheft-ma,  the  strategic  key  to  their  self-defense  is  a deteriorating  system  of  fortifications  built  in  the 
coastal  wetlands  surrounding  their  cities.  But  these  wetlands  are  mysteriously  disappearing  at  an  alarming  rate,  and 
the  threat  of  invasion  by  Mruk-ma  fleets  is  growing. 

A decisive  change  comes  when  the  survey  ship  of  an  advanced  alien  race  crash-lands  in  the  oceans  of 
Mertis.  Arriving  in  escape  pods  from  their  doomed  spaceship,  the  strangers,  called  Cilati,  are  scattered  around  the 
planet.  Now  hopelessly  stranded  on  Mertis,  some  of  the  alien  crew  manages  to  make  their  way  to  The  Coastlands, 
where  they  are  warmly  welcomed  by  the  Sheft-ma.  The  Cilati  survey  team  brings  with  them  precious  scientific 
knowledge,  technology,  and  methods  that  could  dramatically  shift  the  balance  of  power  in  the  conflict  between  the 
two  rival  species.  The  survival  of  the  Sheft-ma  will  depend  on  whether  they  can  effectively  utilize  the  science  and 
tools  of  the  Cilati  to  rebuild  their  crumbling  forts  and  defend  their  disappearing  coastlines. 

The  Celiats  are  a highly  advanced  race  of  space-faring  explorers.  Extremely  long-lived,  they  traverse  the 
galaxy  in  pursuit  of  knowledge  about  other  planets  and  other  life  forms.  Celiat  ships  have  visited  countless  worlds, 
quietly  observing  the  species  that  inhabit  them.  Generally,  they  never  interfere  in  the  cultures  they  study,  and  they 
seldom  even  make  their  presence  known. 

However,  it  quickly  becomes  apparent  that  the  Mruk-ma  have  adopted  a radically  new  strategy  in  their 
struggle  with  the  Sheft-ma:  ecological  warfare.  By  attacking  the  delicate  environment  on  which  their  peaceful  rivals 
depend,  the  Mruk-ma  hope  to  wreck  the  Sheft-ma  civilization  and  eliminate  their  species. 

The  future  of  their  civilization  now  depends  on  whether  the  Sheft-ma  will  be  able  to  master  the  mysterious 
science  of  their  alien  allies,  the  Cilati.  It  is  only  amidst  desperate  circumstances  like  these  that  heroes 

emerge Heroes  who  know  that  understanding  the  world  around  them  is  the  first  step  in  controlling  it. . . .Heroes 

who  recognize  that  well-armed  fleets  and  fortresses  alone  will  not  be  enough.. . .Heroes  who  foresee  that  the  secrets 
of  science  will  be  the  key  to  Conquest  of  the  Coastlands. 

Quest  Learning  Goals 

In  order  to  test  the  analogical  encoding  concept,  we  have  designed  a quest  which  engages  the  player  in 
guided  scientific  inquiry.  The  player  learns  that  Sheftma  fishermen  are  reporting  drastically  smaller  fish  catches  in 
recent  months.  Additionally,  the  player  learns  that  a “monster”  has  been  sighted  by  some  fishermen,  causing  some 
to  believe  the  two  events  are  related.  The  player  character  is  asked  to  research  the  problem. 

This  quest  provides  an  opportunity  for  the  learner  to  explore  an  ecosystem  that  has  been  disturbed  by  an 
invasive  species.  Through  the  investigation,  the  learner  will  discover  that  an  invasive  species  (i.e.  the  monster 
sighted  by  fishermen)  is  indirectly  affecting  the  fish  populations.  In  this  scenario,  the  undisturbed  food  web  consists 
of  the  following:  the  fish  consumes  a shrimp-like  creature  and  the  shrimp-like  creature  consumes  zooplankton. 

Many  fishermen  believe  that  the  monster  is  eating  the  fish.  But  in  actuality,  the  monster  is  an  herbivore 
that  consumes  seaweed  used  by  the  shrimp-like  creature  for  laying  its  eggs.  It  is  the  destruction  of  the  shrimp-like 
creature’s  nursery  that  is  resulting  in  a decline  of  shrimp,  the  sole  food  source  for  the  fish. 

The  player  does  not  work  alone  on  this  problem,  but  is  mentored  by  two  non-player  characters:  a Cilati  and 
a learned  Sheftma  elder.  The  Cilati  provides  support  and  guidance  to  the  player  in  conducting  the  research.  The 
quest  has  the  following  learning  goals: 

• Scientific  Inquiry’:  With  guidance,  learners  are  able  to  articulate  the  question  or  problem  at  hand,  design  an 

investigation,  gather  data,  draw  conclusions,  and  communicate  the  results  and  research  process  (National 

Science  Content  Standard  A). 

• Ecosystems  and  Populations  (National  Science  Content  Standard  C) 

Enduring  Understandings 

o A population  consists  of  all  individuals  of  a species  that  occur  together  at  a given  place  and  time.  All 
populations  living  together  and  the  physical  factors  with  which  they  interact  compose  an  ecosystem. 

o The  equilibrium  of  an  ecosystem  may  change. 

o The  number  of  organisms  an  ecosystem  can  support  depends  on  the  resources  available  and  abiotic 
factors. 
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Skills 

o Identify  the  impact  of  an  organism  on  an  ecosystem 
o Identify  food  webs 

o Observe  and  describe  how  organisms  including  producers,  consumers,  and  decomposers  live  together 
in  an  environment  and  use  existing  resources 
o Classify  living  things  by  common  characteristics 

Through  interactions  with  the  Cilati  mentor,  the  player  character  is  guided  through  defining  the  research 
question  (i.e.  Why  are  the  fish  catches  smaller  than  usual?)  and  conducting  initial  data  collection  on  fish  catches. 
During  this  process  the  player  learns  of  “monster”  sightings  in  the  same  region.  The  player  eventually  visits  the 
fishing  area  to  collect  data.  During  this  site  visit,  the  player  makes  a sighting  of  the  “monster”  that  was  reported 
earlier. 

After  the  player  character  has  seen  the  beast  and  a certain  amount  of  time  has  elapsed,  the  Cilati  mentor  will 
contact  the  player  character  describing  that  he  has  located  a few  relevant  cases  from  another  planet  they  studied  that 
might  be  useful  in  solving  this  problem.  Recall  that  the  Cilati  have  traveled  the  galaxy  in  pursuit  of  knowledge 
about  other  planets  and  other  life  forms.  The  player’s  Library  Tool  (i.e.  analogical  encoding  tool)  automatically 
opens  to  display  two  relevant  cases.  The  Cilati  assigns  the  player  the  task  of  reading  and  comparing  the  two  cases. 

Design  of  the  Cases  and  the  Analogical  Encoding  Tool 

The  cases  presented  by  the  analogical  encoding  tool  are  real  Earth  cases  where  an  ecosystem  has  been 
disturbed  by  an  invasive  species.  The  underlying  principle  that  we  wish  the  player  to  draw  out  is  that  an  invasive 
species  can  disrupt  the  balance  of  an  ecosystem.  The  change  in  the  ecosystem  caused  by  the  invasive  species  may 
impact  a native  species  of  interest  directly  or  indirectly.  Consider  the  two  following  cases: 

Case  1:  Introducing  New  Species  Disrupts  the  Food  Web 

Written  by  Sizon  Matwr 

Cilate  Stardate  11.43.15 

Observation  of  Earth  date  February  15,  1989 

In  the  earth  year,  1965,  homosapiens  on  the  North  American  continent  in  the  state 
of  Montana,  introduced  the  opossum  shrimp  as  a food  source  for  salmon.  The 
shrimp  had  been  identified  as  a food  sources  for  salmon,  but  researchers  did  not 
realize  that  the  invasive  shrimp  ate  the  cladoceran  and  copepod  that  supported 
young  salmon.  Within  a short  period  of  time,  the  shrimp  reduced  the  food 
population  for  the  young  salmon  which  in  turn  impacted  the  number  of  adult 
salmon  and  salmon  predators  such  as  Bald  Eagles  and  Grizzly  Bears.  Through  this 
experience,  homosapiens  learned  that  introducing  a species  can  disrupt  the  entire 
food  web  and  lead  to  a larger  problem  than  they  originally  faced. 

Case  2:  Nutria 

The  nutria,  scientific  name  Myocastor  coypus  of  the  Animal  kingdom,  Phylum 
Chordata,  and  Class  Mammalia  is  found  in  fresh  and  salt  water  ponds  and  swamps. 

It  is  a large  semi-aquatic  rodent  that  is  native  to  South  America.  They  were 
introduced  into  the  state  of  Louisiana  in  North  America  in  1930  and  farmed  for 
their  fur.  Some  nutria  escaped  and  established  breeding  populations  in  areas 
ranging  from  Texas  to  Virginia.  Many  native  birds,  crustacean,  and  fish  species  lay 
their  eggs  in  the  wetlands.  Nutria  are  herbivores  and  devour  native  wetland 
vegetation.  The  nutria’s  disruption  of  the  wetlands  directly  affects  the  life  cycle  of 
many  wetland  organisms. 

As  nutria  destroy  the  native  aquatic  vegetation,  they  remove  plants  and  their  roots 
which  hold  the  soil  in  place.  The  turbidity  of  the  water  increases  because  the  soil  is 
washed  away  in  the  tidal  flow  and  rain.  Increased  turbidity  raises  the  temperature 
of  the  water  and  decreases  the  amount  of  dissolved  oxygen. 
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Disturbing  the  balance  of  the  native  plants  provides  an  advantage  for  non-native 
plant  species  to  become  established. 

Native  animals  and  fish  that  are  dependent  on  the  foliage,  nectar,  pollen,  and  roots 
of  native  plants  are  impacted. 

Attempts  to  control  the  nutria  include  hunting  for  fur  and  food.  Successful  wetland 
restoration  projects  include  fencing  areas  to  exclude  nutria. 

Based  on  the  approach  used  in  the  research  on  analogical  encoding,  our  tool  displays  the  two  cases  side-by- 
side.  Additionally,  the  tool  displays  a grid  with  the  key  aspects  of  case  1 entered  into  the  grid.  The  player  is  then 
asked  to  fill  in  the  aspects  of  case  2 that  are  similar  and  then  write  a conclusion  or  summary  statement. 


Case  1 (Provided) 

Case  2 (Completed  by  the  Player) 

Reduction  in  the  amount  of  Salmon 

Reduction  in  amount  of  Red  Snapper 

opossum  shrimp 

Nutria 

Carnivore  - eats  native  cladoceran  and  copepod 

Herbivore  - eats  native  plants 

Removes  and  disrupts  areas  where  fish  and  shrimp 
lay  eggs  and  young  live  the  first  part  of  their  lives 

Removes  food  required  by  young  salmon 

Reduces  fish  and  shrimp  populations 

Fewer  cladoceran  and  copepod  so  Salmon 
population  is  reduced 

Fewer  shrimp  for  Red  Snapper  so  their  number  is 
reduced 

Conclusion:  Disruption  of  the  habitat  required  for  development  of  organisms  in  a food  web  impacts  the 
rest  of  the  web 

The  tool,  once  fully  implemented  in  the  virtual  learning  environment,  will  enable  the  player  to  drag  and 
drop  key  phrases  from  case  2 into  the  grid.  This  will  enable  us  to  provide  more  accurate  feedback  to  the  player. 

Future  Research  and  Development 

During  Spring  2007,  we  will  conduct  a pilot  research  study  to  explore  the  impact  of  the  analogical 
encoding  tool  on  student  learning.  We  will  compare  experimental  (access  to  the  analogical  encoding  tool)  and 
control  conditions  (do  not  have  access  to  the  analogical  encoding  tool).  It  is  our  expectation  that  once  the  tool  is 
refined  we  will  expand  its  application  so  that  the  tool  shows  relevant  cases  for  numerous  contexts  in  the  game. 
Additionally,  we  are  interested  in  exploring  different  types  of  support  that  can  be  provided  in  the  tool.  For  example, 
we  may  examine  various  strategies  such  as  asking  students  to  describe  similarities  and  differences,  providing 
questions  that  scaffold  comparison  of  the  cases,  and  providing  intensive  analogy  training  (Kurtz,  Miao,  and  Gentner, 
2001). 
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Transformative  Learning  Experiences:  How  Do  We  Get  Students  Deeply 

Engaged  for  Lasting  Change? 

Brent  G.  Wilson 
Scott  H.  Switzer 
Patrick  Parrish 
The  IDEAL  Research  Lab 

In  today's  world  of  generic  instructional  methods  and  standards,  we  have  observed  a need  for 
deeper  kinds  of  learning  engagement  - the  kind  that  leaves  a lasting  impression  on  the  learner  and 
has  a demonstrable  impact  on  practices.  In  this  paper  we  offer  our  initial  model  for  achieving 
transformative  learning  through  deep  engagement.  The  model  draws  on  theories  of  cognition, 
aesthetics,  and  ritual  and  symbolic  support  for  identity  change. 

For  the  past  two  years  our  research  lab  has  explored  ways  to  broaden  the  theoretical  foundations  of 
instructional  design  (ID),  particularly  to  strengthen  the  relevance  of  ID  thinking  to  problems  encountered  in  practice. 
Our  recent  work  has  explored  how  practicing  instructional  designers  follow  aesthetic  concepts  in  designing  learning 
experiences  (Parrish,  2005,  Parrish,  2006)  and  how  ID  practices  fit  within  larger  organizational  values  of  respecting 
diverse  learners  and  encouraging  inclusive  practices  (Wilson,  2005a,  Wilson,  2005b). 

One  problem  we  have  observed  is  the  proliferation  of  generic,  highly-templated  learning  resources.  As 
traditional  training  materials  are  converted  to  e-learning  in  particular,  high  volumes  of  content  are  produced  but 
often  at  a sacrifice  to  quality  (Merrill  & Wilson,  2007).  In  higher  education  and  K12  environments,  curriculum  is 
referenced  to  standards,  but  learning  activities  often  do  not  push  beyond  a superficial  level.  In  these  technically 
correct  but  shallow  learning  environments,  learners  will  express  a need  for  deeper  kinds  of  engagement  - the  kind 
that  leaves  a lasting  impression  and  has  a demonstrable  impact  on  their  practices.  In  this  paper  we  offer  an  initial 
analysis  for  understanding  these  concerns,  and  point  the  way  for  continued  inquiry  along  these  lines. 

Transformative  Learning  - What  is  it  and  Why  Should  We  Care? 

Instruction  comes  in  different  forms  - tutorials,  simulations,  courses,  workshops.  Some  instruction  is  designed  to  be 
short,  simple,  and  directly  tied  to  performing  a required  work  task;  other  times  instruction  may  be  quite  removed 
from  the  immediate  context  of  a practical  task.  One  important  kind  of  instruction  is  the  kind  that  leaves  a deep  and 
lasting  impression  on  the  learner.  Instruction  can  transform  the  learner  in  different  ways,  e.g.,  how  the  learner  sees 
herself  - as  a professional,  a learner,  or  problem  solver  of  some  kind.  Or  how  the  learner  sees  the  subject  matter, 
perhaps  leading  to  an  entirely  new  way  of  seeing  problems  and  situations.  Typically  this  happens  in  larger  units  of 
instruction,  e.g.,  full  program  curriculum,  semester-length  courses,  or  week-long  workshops.  Striking  impressions 
can  be  made  in  shorter  forms,  but  would  be  less  expected  for  narrow,  more  technical  kinds  of  training. 

The  term  transformative  learning  is  used  by  Mezirow  and  associates  (e.g.,  Mezirow,  1991;  also  Cranston  & 
King,  2003;  Dirkx,  2000)  as  an  approach  to  adult  education  that  emphasizes  its  emancipatory  potential  and  enhances 
awareness  of  social  responsibility.  Our  use  of  the  term  is  somewhat  different.  In  contrast  to  more  routine  kinds  of 
learning  outcomes,  transformative  learning  would  seem  to  involve  the  following  indicators: 

• Significant  growth  in  the  learner's  identity  as  a competent  problem  solver  or  practitioner.  The  learner 
becomes  a constructive  participant  as  opposed  to  a consumer  or  recipient  of  content. 

• Significant  restructuring  of  learners'  schemas  about  the  subject  matter.  The  learner  develops  a new  stance 
toward  the  subject  matter. 

• Positive  shift  in  interest,  commitment,  values,  or  attitudes  toward  the  subject  matter.  The  learner  develops 
new  intentions  or  plans  for  action  based  on  their  new  understanding  (Hidi  & Renninger,  2006). 

• Strong  feelings  toward  an  instructor  or  learning  peers,  who  may  serve  as  guides,  models,  mentors,  or 
inspiration  in  the  learning  process.  The  learner  develops  new,  reciprocally  rewarding  relationships  with 
teachers/leaders,  learners,  or  a community  of  practice. 

In  other  words,  transformative  learning  is  understood  in  terms  of  its  cognitive,  attitudinal,  identity-forming,  or 
motivational  impacts,  as  reported  by  learners  themselves.  While  all  learning  may  be  seen  to  transform  the  learner  in 
some  way,  our  use  of  the  term  denotes  a deeper  or  more  impacting  kind  of  change  as  suggested  by  the  indicators 
above.  While  all  learning  has  some  value,  we  would  argue  that  transformative  learning  plays  a unique  role  in 
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shaping  learners'  identity  and  aspirations,  which  in  turn  lead  to  commitments  and  action  plans.  In  other  words, 
transformative  learning  experiences  can  be  pivotal  or  defining  moments  in  learners'  lives. 

Transformative  Learning  Experience 

We  think  of  transformative  learning  experience  as  the  learner's  construction  of  learning  events  that  were  particularly 
meaningful  and  transforming  in  their  lives.  A transformative  learning  experience  (TLE)  is  defined  as  having  the 
following  qualities: 

• Lasting  impression.  The  learner  holds  in  memory  details  about  the  learning  experience 

• Part  of  the  person 's  self-narrative.  The  learner  references  the  learning  experience  within  a narrative  about 
themselves  or  their  relation  to  a subject  matter  of  importance  to  them. 

• Behavioral  impact.  The  learner  can  point  to  specific  changes  in  their  lives  as  a result  of  the  learning 
experience. 

These  qualities  all  point  to  the  constructed  nature  of  the  experience.  This  is  consistent  with  the  notion  of 
experience  in  general.  Experience,  we  believe,  is  best  seen  from  the  learner's  point  of  view.  Table  1 relates 
experience  to  instructional  strategies  and  learning  activities,  two  other  constructs  used  to  explain  learning  within  the 
ID  literature. 

Table  1. 

Four  ways  to  thinking  about  activities  that  lead  to  learning. 


Way  of  Thinking 
About  Activity 

Description 

Intentional 

Perspective 

Learning  experience 

The  learning  activity  and  its  significance  as  perceived  by  the 

Learner 

learner,  containing 

(Designers  or 

Narrative  - what  happened  when 

teachers  may 

Lesson  or  outcome  - The  point,  outcome,  or  lesson 

also  think  of 

learned  from  of  the  narrative 

designing  an 
experience) 

Learning  activity 

Description  of  activities  that  students  engage  in  for  learning 

Teacher  or 
learner 

Enacted  instructional 

The  adaptations  and  improvisations  needed  to  make  the 

Teacher 

strategy > 

strategy  work  in  real  time 

Planned  instructional 

A specific  type  of  learning  activity  from  an  instructional 

Teacher  or 

strategy’ 

theory  or  curriculum  guide 

curriculum 

designer 

We  categories  above  are  not  organized  into  a hierarchy,  that  is,  learner  experience  is  not  intended  to  be  privileged 
above  strategies  or  activities.  Indeed,  learning  activities  can  arguably  be  most  closed  connected  causally  to  learning 
outcomes  (Ryder,  2006).  Strategies  are  helpful  when  understanding  teacher  or  designer  intentions;  activities  have 
the  clearest  link  to  learning;  experiences,  we  believe,  are  valuable  in  what  they  tell  us  about  learner  perceptions  and 
schemas,  and  the  lessons  they  take  away  from  instruction. 

Consistent  with  this  constructed  view  of  learner  experience,  we  can  see  immediately  that  individual 
learners  contribute  substantially  to  the  process.  It's  not  so  much  about  what  is  "done"  to  the  learner  as  how  the 
learner  participates  in  the  experience.  Teachers  and  designers  can  talk  about  crafting  or  designing  the  learner 
experience,  but  the  level  of  control  over  that  learner  experience  is  quite  indirect  and  mediated  by  several  factors  not 
under  teacher  control.  With  respect  to  transformative  learning,  some  learners  may  be  ready  for  deeper  change  while 
others  are  not.  Some  may  attach  to  an  instructor  as  a role  model  or  inspiration,  while  others  don't.  So  part  of  the 
discussion  needs  to  examine  the  relationship  between  learner  and  instructor,  or  between  learner  and  content  - not 
just  the  features  of  instruction  alone. 

The  construct  of  transformative  learning  experience  is  very  close  to  Gordon  Rowland’s  notion  of  powerful 
learning  experience,  defined  as  “an  experience  that  stands  out  in  memory  because  of  its  high  quality,  impact  on 
one’s  thoughts  and  actions  over  time,  and  transfer  to  a wide  variety  of  contexts  and  circumstances  (Rowland, 
Lederhouse,  & Saterfield,  2004;  see  also  Rowland  & DiVasto,  2001;  Rowland,  Hetherington,  & Raasch,  2002). 
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Encountering  Rowland’s  work  late  in  the  conceptual  development  of  the  construct,  we  will  attempt  to  better 
integrate  Rowland’s  work  with  our  own  in  a future  paper. 

To  summarize,  we  see  transformative  learning  experience  or  TLE  in  these  terms: 

• It  is  related  to  strategies  or  methods  of  instruction,  but  is  not  determined  by  those  methods. 

• It  has  to  do  with  instructor  and  peer  relationships,  but  is  not  determined  by  those  relationships. 

• It  may  be  a targeted  learning  goal  of  instruction,  but  cannot  be  imposed  on  all  learners,  no  matter  how  rich 
the  instruction.  Learner  readiness  comes  into  play. 

• It  leads  to  cognitive  growth,  but  also  attitudinal,  identity,  and  action/agency  change. 

• It  is  fundamentally  determined  by  learner  perceptions. 

• It  may  be  re-evaluated  or  appreciated  differentially  over  time.  That  is,  a transformative  learning  experience 
may  not  immediately  be  apprehended,  but  over  time  come  to  be  seen  as  transformative.  Thus  a 
transformative  learning  experience  becomes  part  of  a person's  self-narrative,  which  may  change  over  time. 
To  help  learners  achieve  deeper  transformations,  great  care  must  be  taken  to  carefully  shape  and  craft  the 

learning  encounter  - the  materials,  the  sequencing,  the  interactions,  the  symbols,  the  content.  This  stance  toward  ID 
practice  stands  in  contrast  to  strategy-centric  beliefs  - relying  on  the  choice  of  the  right  instructional  strategy  to 
make  the  learning  difference.  In  the  best  conditions,  designers  take  the  time  to  craft  and  hone  the  environment  and 
resources  to  achieved  desired  ends  - in  this  case,  deeper  engagement  with  a hope  of  learner  transformation.  In  the 
discussion  below  we  present  three  layers  or  lenses  for  viewing  instruction  - cognitive,  aesthetic,  and  mythic 
perspectives.  We  argue  that  to  achieve  deeper  levels  of  learner  engagement  and  transformative  change,  designers 
should  attend  to  all  three  layers  of  design  as  they  visit  and  re-visit  their  learning  resources,  always  with  an  aim  to 
heighten  the  learning  experience. 

Cognitive  Perspective  - TLE  as  Effective  Learning  Program 

Most  practicing  designers  trained  in  current  graduate  programs  approach  designs  from  a constructivist  or  pragmatic 
theoretical  stance.  Our  first,  "native"  approach  is  to  ask,  how  can  I design  a learning  activity  that  addresses  the 
required  standards  and  objectives,  but  gives  learners  a chance  to  actively  make  things  and  construct  meaning?  A 
good  constructivist  approach  to  instruction  would  be: 

• Complex,  problem-based 

• Authentic,  realistic 

• Scaffolded,  guided 

• Active,  participatory 

• Reflective,  intentional 

We  do  not  elaborate  on  this  cognitive  level  of  analysis  because  it  currently  predominates  ID  thinking,  both  in 
literature  and  in  practice  (cf.  Duffy  & Jonassen,  1992;  Dunlap  & Grabinger,  1996).  A variety  of  instructional 
strategies  are  consistent  with  current  cognitive  learning  theory,  including  problem-based,  project-based,  and  guided- 
inquiry  learning. 

Cognitive  load  theory  represents  another  approach  to  cognitive  learning  theory  (Kirschner,  Sweller,  & 
Clark,  2006;  Mayer,  2004;  Paas,  Renkl,  & Sweller,  2003;  Van  Merrienboer,  Kirschner,  & Kester,  2003).  We  see 
cognitive  load  considerations  more  specifically  addressing  the  ongoing  thinking  and  processing  while  students  are 
engaged  in  learning  activities  - a micro  level  of  analysis.  Thus  once  a constructivist  activity  has  been  developed, 
designers  should  examine  the  information-processing  requirements  of  the  lessons  - in  particular  the  completeness 
and  availability  of  information  for  task  completion.  This  micro-level  analysis  focuses  on  moment-to-moment 
cognition  and  complements  the  more  holistic  search  for  a constructivist  activity  that  would  drive  the  module  or  unit. 
Materials  can  then  be  fine-tuned  to  respect  cognitive  load  constraints  and  provide  adequate  guidance  and  direction  to 
students  within  the  overall  constructivist  environment. 

Constructivist  learning  activities  and  cognitive  load  analyses  - these  are  just  two  methods  for  informing 
instructional  designs  from  a cognitive  perspective.  The  cognitive  layer  of  design  should  be  open  to  all  the  thinking 
currently  in  place  informed  by  cognitive  psychology.  Indeed  each  layer  of  design  - cognitive,  aesthetic,  and  mythic 
- has  a unique  contribution  in  the  overall  effort.  The  main  value  of  the  cognitive  perspective  lies  in  its  integration 
into  present  practices,  making  it  a natural  starting  point  for  most  of  us,  and  in  its  close  connection  to  psychology  and 
learning  research,  which  is  steadily  advancing  understanding  of  cognition  and  learning.  We  would  point  out, 
however,  that  current  cognitive  models  are  better  at  representing  a problem  space  than  at  pinpointing  where  problem 
solutions  come  from  (Schon,  1990).  Once  a solution  possibility  comes  to  mind,  known  testing  mechanisms  can  kick 
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in  - designers  can  rationally  evaluate  the  design  space  and  determine  an  appropriate  strategy.  Where  those  creative 
solutions  come  from  is  less  well  understood.  Because  transformative  learning  experiences  require  unusually  creative 
designs,  we  are  well  advised  to  rely  less  on  narrow  cognitive  formulations  of  the  design  space.  Aesthetic  and  mythic 
perspectives  on  the  learner  experience,  both  of  which  stretch  designers  toward  more  creative  thinking,  are  presented 
below. 

Aesthetic  Perspective  - TLE  as  Intensive  Encounter 

This  perspective  asks  designers  to  think  of  instruction  as  an  aesthetic  encounter  - much  like  watching  a movie  or 
listening  to  a symphony.  Countering  tendencies  toward  exceptionality  among  romantic  and  rationalistic  thinkers, 
Dewey  (1934)  articulated  a pragmatic  view  of  aesthetics  that  brought  it  into  people's  everyday  life,  as  intensified 
immediate  experience  (see  also  Berleant,  1991).  A number  of  aesthetic  motifs  come  into  play  when  we  examine  the 
kind  of  intensive  instruction  that  would  lead  to  transformative  learning  experience: 

• Narrative  or  storytelling.  Some  of  the  most  powerful  works  of  art  depend  on  narrative  as  a central 
organizing  device.  Similarly  instruction  can  be  thought  of  as  a story  where  the  learner,  instructor,  and 
classmates  become  participants  or  actors. 

• Dramatic  tension.  Stories  depend  on  people's  schemas  and  expectations.  Moreover  they  build  levels  of 
dramatic  tension  by  unfolding  events  that  disrupt  a harmony  or  peaceful  situation.  Stories  both  depend  on 
and  diverge  from  established  expectations,  leading  to  a sense  of  anticipation  and  desired  fulfillment. 
Instructional  disruptions  - in  the  form  of  novel  events,  complex  problems,  and  challenges  - can  be  useful  to 
learning  if  they  are  carefully  managed  and  the  learner  is  brought  through  the  disruption  in  a positive  way. 
Conflict  and  resolution  are  core  narrative  devices  that  also  play  a role  in  powerful  learning  experiences. 
Managing  the  dramatic  tension  requires  careful  attention  to  the  pacing  and  rhythm  of  unfolding  events  and 
challenges  - too  much  novelty  can  alienate  the  participant,  too  little  change  can  lead  to  boredom  and 
inattention.  Careful  and  effective  pacing  not  only  helps  with  arousal  levels,  but  buy-in  and  commitment  as 
well. 

• Resolution  and  closure.  All  narrative  forms  of  art  include  some  kind  of  resolution  of  tension  that  leads  to  a 
satisfying  closure  at  the  end  of  the  experience.  While  not  always  a “happy  ending,”  closure  brings  things 
together  in  a way  that  we  can  see  how  the  story  elements  fit  together  into  a meaningful  whole.  The  more 
control  the  author/artist  has  over  the  elements  of  the  narrative,  the  stronger  the  satisfaction  and  appreciation 
at  point  of  closure.  Instructional  programs  also  need  clear  forms  of  closure,  signaling  to  learners  a variety 
of  things  - end  of  program,  consolidation  of  understanding,  time  for  reflection  and  apprehension  of 
meaning  from  the  experience,  preparation  for  next  steps  such  as  taking  the  next  course  or  entry  into  the 
world  as  a newly  skilled  person.  More  attention  to  the  closure  experience  can  help  learners  draw  more 
meaning  from  the  instmction,  which  would  in  turn  make  them  more  likely  to  use  their  new  knowledge 
more  appropriately  when  it’s  needed. 

• Empathy,  role  identification,  and  perspective  taking.  Learners  build  relationships  with  other  course 
participants,  including  the  instructor.  They  must  come  to  see  problems  through  the  eyes  of  others.  They 
sometimes  take  on  new  roles  and  identities,  sometimes  temporary  but  sometimes  required  by  the  subject 
matter.  At  times  learners  are  called  to  engage  in  a suspension  of  disbelief,  or  participate  as  if  certain 
conditions  were  true.  All  of  these  projective  elements  are  similar  to  how  people  respond  to  works  of  art. 
They  help  learners  participants  are  complete,  whole-person  levels  that  include  emotions  and  self- 
constructions. For  transformative  learning  to  occur,  this  kind  of  engagement  seems  critical. 

• Production  details.  Designers  usually  think  of  aesthetics  in  terms  of  production  values,  attractiveness,  and 
aesthetic  appeal.  Typography,  color,  media  design  and  images  - these  also  contribute  to  a learner's 
apprehension  of  the  immediate  stimulus  or  activity  and  can  help  it  rise  to  a more  intense  level  of 
experience.  We  may  also  include  here  some  of  the  attention-grabbing  design  ideas  of  Kathy  Sierra  (2006), 
which  includes  visual  design  but  also  involves  style  and  content  elements  designed  to  engage  and  interest 
the  learner. 

An  aesthetic  perspective  on  instruction  could  become  a primary  basis  for  organizing  a course  or  curriculum. 
More  commonly,  though,  designers  will  take  existing  or  planned  learning  activities,  and  re-examine  them  through  an 
aesthetic  lens  as  a means  to  craft  the  activities  into  a more  elegant  and  satisfying  learning  experience.  When 
deciding  on  the  pacing  for  a course,  for  example,  a designer  may  carefully  sequence  activities  and  assignments  in  a 
way  that  creates  cycles  of  tension  and  resolution  - with  students  carefully  led  through  challenges  and  peaks  of 
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engagement,  followed  by  consolidation  and  recouping  energy  reserves,  culminating  in  a final  challenge  and 
resolution  at  the  end. 

Parrish  (in  press)  has  developed  a method  for  plotting  learner  engagement  in  a course,  based  on  existing 
methods  for  depicting  dramatic  tension  in  novels  and  stories.  Figure  1 shows  the  projected  pacing  and  flow  of  a 
typical  course. 


End 
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Figure  1:  Engagement  curve  of  a typical  course  (from  Parrish,  in  press). 

The  vertical  axis  is  a combination  of  learner  engagement  and  overall  complexity,  thus  accounting  for  the  overall 
upward  slope  of  the  line.  Learners  become  engaged  in  response  to  the  central  challenges  of  the  course.  They 
maintain,  with  some  ups  and  downs,  a level  of  engagement  as  they  persevere  through  the  middle  sections,  which 
have  the  potential  to  lose  engagement  if  not  well  designed.  Engagement  typically  peaks  toward  the  course's 
conclusion,  with  a resolution  of  some  kind  achieved  at  the  end. 

The  plot  below  (Figure  2)  projects  engagement  and  complexity  for  an  instructional-design  course  that 
compares  two  instructional  models  and  has  students  develop  and  critique  each  other's  design  projects.  Following  this 
plotting,  a level  of  tension  is  introduced  as  learners  confront  differences  in  two  instructional  models,  leading  to 
reflection  and  growth  about  the  process  in  general. 
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Students  analyze  the 
two  contrasting 
instructional 
approaches  for  their 
differing  values  and 
assumptions,  and  for 
their  efficacy  in 
addressing  the  goal  in 
the  stated  situation. 


Students 
identify  an 
instructional 
goal  and  apply 
their  chosen 
model  to 
addressing  it. 


Students  further 
develop  their 
design  and  do 
formative 
evaluation  ■ 


Students  explore 
the  theory  behind 
an  instructional 
model  and  apply  it 
to  an  instructional 
goal. 


Students  present 
their  design, 
justify  and  critique 
it  for  the  class 


Students  critique 
each  other's 
design 
documents. 


Students  explore  a 
contrasting 
instructional 
model,  and  then 
apply  it  to  the 
same  instructional 
goal  as  done  with 
the  first  model. 


Further  exploration  of 
other  instructional 
models,  related  to  their 
goals  and  assumptions. 
Students  look  for  a 
model  they  want  to  work 
with  for  their  project. 


Figure  2:  An  engagement  curve  used  for  planning  a course  in  instructional  models  for  instructional 
designers  (from  Parrish,  in  press). 
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Parrish  (2006)  includes  other  methods  of  diagramming  the  learning  experience.  While  these  tools  will 
likely  be  useful  in  some  situations,  the  figures  above  are  intended  to  illustrate  ways  to  interpret  instruction  from  an 
aesthetic  perspective,  which  could  lead  to  added  insights  about  the  learner's  incoming  experience. 

Mythic  Perspective  - TLE  as  a Hero's  Journey 

The  mythic  perspective  is  similar  to  the  aesthetic  in  its  reliance  on  narrative  as  a central  organizing  device. 
Instruction  becomes  mythic  when  the  learner  identifies  as  a participant  engaged  in  a life -relevant  journey  or  path 
that  will  lead  to  conquered  challenges  and  new  identity.  Insights  into  mythic  processes  are  drawn  from  anthropology 
and  comparative  religion  (e.g.,  Bateson,  1975;  Campbell,  1968;  Ileade,  1954;  Jung,  1968;  Trice  & Beyer,  1984; 
Tripp,  1994;  see  also  Groen  & Jacob,  2006). 

• Immersive  journey.  Joseph  Campbell's  (1968)  synthesizing  work  on  archetypal  figures  in  world  religions 
increased  awareness  of  the  hero's  journey.  The  hero  must  leave  home  and  go  out  into  the  world  to  find  his 
fortune,  overcoming  challenges  and  returning  home  with  gifts.  In  the  course  of  doing  so  the  hero  undergoes 
deep  and  transformative  change,  including  often  a new  name.  Parts  of  the  journey  may  involve  immersion, 
flow,  or  intoxication,  as  well  as  more  rational/intentions  forms  of  participation. 

• Rituals  and  symbols.  Life  events  of  community  members  are  endowed  with  meaning  through  various  rites, 
symbols,  and  language  forms.  These  symbols  serve  to  convey  continuity  and  order  as  people  progress 
through  various  changes  and  transformations  in  their  lives  (e.g.,  new  names,  investitures,  personas;  rites  of 
passage,  inclusion,  and  achievement).  In  some  induction  programs  such  as  the  military  or  teacher  training, 
we  see  signs  of  attention  to  these  mythical  aspects  of  the  change  process.  Certainly  these  symbols  are 
evident  more  broadly  in  society  at  large,  in  both  indigenous  and  Western  cultures  (Reagan,  2000,  pp.  25- 
55);  Trice  & Beyer,  1984).  Instructional  programs  aspiring  to  transform  learner  identities  should  look  for 
symbols  and  language  that  could  signify  major  life  changes,  giving  participants  a sense  of  meaningfulness 
and  belonging. 

• Teacher  as  master  guide.  Joseph  Campbell  (1968)  speaks  of  an  archetypal  guide  that  is  “protective  and 
dangerous,  motherly  and  fatherly  at  the  same  time”  (p.  73).  Mentors  or  masters  can  be  tremendously 
important  not  just  to  learning  but  to  personal  growth  and  maturation.  In  Western  history,  the  role  of  master 
teacher  has  been  appreciated  since  the  days  of  Socrates,  and  has  even  involved  a sexual  element  (Steiner, 
2003).  India  has  the  tradition  of  gurus  as  guides.  Over  time  and  through  occasional  twists  or  ritual 
processes,  learners  and  apprentices  take  on  the  identity  and  qualities  of  the  master  and  establish  themselves 
as  masters  in  their  own  right.  Transformative  instruction  will  rely  on  a skilled  master  teacher — not  just  a 
conveyer  of  content  but  a “protective  and  dangerous”  guide — and  may  sometimes  provide  a mechanism  to 
challenge  and  supplant  the  master  (Herman  & Mitchell,  2004). 

• Trust,  risk,  and  transgression.  Hero  journeys  required  an  element  of  life-risk  as  the  protagonists  leaves  the 
comfort  of  the  known  and  enters  more  threatening,  unstable  worlds.  There  is  sometimes  an  element  of 
transgression,  where  breaking  established  norms  and  rules  is  required  to  obtain  hidden  knowledge. 

Likewise  instructional  programs  should  send  learners  permission  to  transgress  and  go  beyond  what  they 
were  taught,  in  order  to  advance  knowledge  and  engage  in  life-long  learning. 

• Reconciliation  and  rapprochement.  Often  in  the  process  of  pursuing  knowledge,  relationships  are 
threatened  and  challenged,  forcing  re-definition  or  re-negotiation  between  people.  This  impact  on 
relationships  is  illustrated  in  films  such  as  Pygmalion  or  Educating  Rita.  Just  as  transgression  and  re- 
invention  may  be  needed  processes,  so  are  processes  of  renewal  and  reconciliation.  Within  instruction  this 
may  extend  to  teacher-student  relations,  or  to  relationships  in  the  world.  Students  need  ways  to  maintain 
and  continue  levels  of  trust  and  buy-in,  even  when  things  go  wrong  or  need  repair. 

Mythic  considerations,  like  aesthetic  ones,  would  often  involve  a secondary  review.  The  designer  may  take  a 
planned  activity  or  set  of  activities,  and  then  look  for  ways  to  deepen  the  experience  through  rites,  symbols,  and 
special  language  - all  with  the  intention  to  increasing  the  depth  of  engagement,  buy-in,  and  role  identification  by  the 
learner. 

A Walk-Through  of  the  Three  Layers 

We  present  below  an  illustrative  walk-through  of  how  these  ideas  could  be  used  to  design  and  deliver  a graduate 
course  in  managing  instructional  projects.  The  details  of  the  course  are  factual,  since  it  was  taught  successfully  by 
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Scott  Switzer  last  year.  We  use  the  course  as  illustration  because,  on  reflection,  it  embodies  many  of  the  aesthetic 
and  mythic  qualities  we  are  looking  for  - even  though  Scott  was  only  partly  aware  of  these  issues  at  the  time. 

Walk-Through  - Cognitive  Layer 

Along  with  the  final  portfolio,  the  management  course  is  intended  as  a capstone  event  for  the  master's  program  in 
instructional  technology.  In  past  offerings  of  the  course,  learners  have  focused  on  learning  the  fundamentals  of 
project  management,  often  through  an  applied  project  of  some  kind.  Program  leaders  saw  a problem  trying  to 
complete  a real-life  project  the  final  semester,  but  were  reluctant  to  give  up  the  authentic  experience.  They  asked 
Scott  to  conceive  of  a way  to  give  learners  an  authentic  management  experience  while  bounded  within  the 
constraints  of  their  final  semester  in  the  program. 

Scott  approached  the  course  using  four  key  principles  culled  directly  from  constructivist  learning  theory 
(Switzer,  2006): 

• The  use  of  an  ill-structured  problem  or  challenge 

• Embedding  this  problem  in  an  authentic  context  (to  the  extent  possible  given  the  parameters  of  an  academic 
semester) 

• The  use  of  focused  reflective  activities  in  the  learning  context 

• The  establishment  of  a mentoring  relationship  between  the  course  facilitator  and  the  learners 
Following  these  principles,  a central  metaphor  for  the  course  was  devised  - having  students  respond  in  teams  to  a 
fictitious  RFP  for  instructional-design  services.  We  present  below  Scott's  design  for  the  course,  which  were  based 
primarily  on  cognitive  instructional  strategies. 

From  the  first  day,  learners  were  presented  with  the  challenge  of  responding  to  a realistic  RFP,  one  that  a 
learner  described  as  “characteristically  vague”  and  typical  of  her  professional  experience  as  a practicing 
instructional  designer.  The  entire  course  was  set  up  as  a competition  where  the  learners,  working  in  teams,  would 
respond  to  this  RFP  for  a fictitious  client.  At  the  initial  meeting,  learners  were  told  that  they  would  be  working  for  a 
client  and  that  indeed,  a winning  bid  would  be  selected  at  the  end  of  the  course.  Learners  were  also  informed  that 
they  would  be  responsible  for  driving  the  course  schedule,  arranging  meetings  with  clients  to  ask  clarifying 
questions,  scheduling  interim  milestones  to  secure  approval  on  their  plan  prior  to  moving  to  the  next  stage  of  the 
design,  and  arranging  for  their  own  support  for  preparing  for  these  meetings.  Class  meetings  were  presented  as  an 
option  for  three  different  events: 

• Formal  client  meetings  where  questions  would  be  clarified,  plans  would  be  presented  and  feedback 
received 

• Class  sessions  to  cover  a variety  of  content  that  learners  would  need  to  succeed  (project  management  skills, 
budgeting,  staffing,  program  evaluation,  and  return  on  investment  issues) 

• Class  sessions  that  provided  an  opportunity  for  thinking  and  reflecting  about  other  issues  that  were  less 
skill-based  (client  management  issues,  being  resourceful,  managing  internal  team  concerns,  etc.)  but  no  less 
important  to  the  success  of  the  program 

The  intent  of  these  sessions  was  to  get  learners  engaged  in  a realistic  process  for  managing  their  own  work. 
Learners  were  aware  that  the  winning  bid  would  be  rewarded  on  a certain  date,  but  in  their  teams,  they  were 
required  to  manage  their  own  work  to  be  able  to  present  their  final  work  to  the  client  by  the  required  date. 

To  ensure  that  learners  did  not  get  bogged  down  with  prerequisite  or  peripheral  knowledge,  critical  content 
was  provided  in  a way  that  supported  the  learning  objectives.  Conducting  a cognitive  load  analysis  demonstrates  a 
respect  for  the  cognitive  complexities  of  a learning  experience  of  this  nature  and  yielded  some  valuable  insight  on 
how  to  manage  those  complexities.  An  example  was  providing  content  resources  for  learning  Microsoft  Project. 

One  of  the  identified  hard  skills  to  be  learned  was  to  develop  a comprehensive  project  management  plan  to  present 
to  clients.  It  was  not  assumed  that  learners  had  previous  experience  with  a project  management  tool  such  as  Project, 
but  it  was  also  clear  that  there  was  not  considerable  time  in  our  scheduled  sessions  for  in-depth  lessons  on  the  tool. 
As  a result,  Adobe  Captivate  was  used  to  create  Flash-based  mini-lessons  that  covered  Project  essentials.  Learners 
could  then  access  the  lessons  as  needed  outside  of  class. 

Another  response  to  cognitive  load  was  to  place  corporate  documents  from  the  fictitious  company  in  a 
common  place  for  learners  or  teams  to  explore  as  they  saw  fit.  Each  document  provided  to  the  portal  had  some 
valuable  information  that  was  important  to  understanding  the  ultimate  client  solution,  but  it  was  not  considered 
required  reading.  Adding  to  the  authenticity  of  the  situation,  learners  were  made  aware  that  additional  information 
was  available  to  them,  but  they  were  ultimately  given  the  responsibility  to  determine  whether  or  not  it  was  a good 
use  of  their  time.  Some  of  the  teams  consumed  all  of  the  information  they  possibly  could,  and  in  most  cases,  there 
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was  an  increased  level  of  insight  to  their  response  to  the  client  training  problem.  Teams  that  initially  did  not  bother 
to  read  all  of  the  documentation  were  compelled  to  re-think  their  decisions  when  some  of  this  information  emerged 
as  part  of  the  client  meeting  sessions.  For  at  least  one  team,  there  was  an  important  lesson  learned  about  not  doing 
your  homework  prior  to  meeting  with  the  client  for  the  first  time.  And  while  this  was  an  unintended  outcome  of  the 
process,  it  was  clear  that  the  experience  was  taken  to  heart:  for  the  remainder  of  the  course,  the  team  never  failed  to 
be  prepared  for  meetings. 

Walk-Through  - Aesthetic  Layer 

An  aesthetic  perspective  requires  careful  attention  to  all  the  design  elements  and  their  configuration  to  achieve  an 
intense,  memorable  experience.  For  this  review  we  use  a framework  established  by  Parrish  (2006),  containing  the 
following  four  elements: 

• Engagement 

• Anticipation 

• Pacing 

• Resolution  or  Closure 

Parrish  (2006)  defines  engagement  as  “a  relationship  to  groups  or  events  in  which  one  willingly  makes  a 
contribution  that  is  active  and  constitutive.  Engagement  involves  an  investment  of  effort  and  emotion,  willingness  to 
risk,  and  concern  about  outcomes”  (p.  1).  In  Scott's  management  course,  learners  were  asked  to  participate  in  ways 
that  are  uncommon  in  traditional  classroom  learning  experience.  By  setting  up  the  environment  as  a team 
competition,  learners  had  a much  greater  stake  in  the  outcome  - early  on,  one  of  the  participants  wondered  aloud 
that  if  there  were  four  teams  in  the  experience,  the  winning  team  would  be  awarded  an  A grade,  the  runner-up  a B, 
and  so  on.  While  Scott  purposefully  never  gave  credence  to  that  assumption,  the  resulting  level  of  intensity  was 
uncommon,  as  evidenced  by  the  considerable  investment  of  time  and  emotion  for  many  groups.  By  the  end  of  the 
second  week,  the  facilitator  could  sense  an  increased  sense  of  urgency  on  the  part  of  most  teams,  along  with  an 
increased  level  of  encouragement  of  teammates  who  felt  lost  or  overwhelmed  by  the  challenge  they  faced.  At  the 
end  of  the  course,  with  one  notable  exception,  all  of  the  learners  were  actively  engaged  in  the  course,  providing 
regular  feedback  in  front  of  the  entire  class  along  the  lines  of  “Eve  never  worked  this  hard  on  any  course!” 

Anticipation  “involves  a concern  for  reaching  a conclusion  or  resolving  a question,  issue,  or  mystery. 
Anticipation  involves  a look  ahead  with  interest  in  outcomes”  (Parrish,  2006,  p.  1).  Initially,  the  intent  was  to 
provide  learners  with  a suitably  vague  RFP,  allow  them  to  engage  in  the  process  of  deconstructing  the  document, 
and  to  support  them  to  consider  all  of  the  possibilities  as  they  went  about  the  business  to  crafting  an  appropriate 
response  to  the  stated  training  problem.  The  competitive  bid  response  to  the  RFP  created  a fairly  strong  sense  of 
anticipation  and  important  consequence. 

The  pacing  of  the  course  is  described  as  “our  perception  of  time  and  the  rate  in  which  events  take  place  in 
time  as  related  to  our  level  of  comfort  and  ability  to  grasp  their  implications  to  a degree  appropriate  to  our  level  of 
interest”  (Parrish,  2006,  p.  1).  As  previously  stated,  the  course  was  initially  framed  as  loosely  as  possible,  placing 
the  onus  of  scheduling  and  preparation  for  the  task  at  hand  directly  in  the  hands  of  the  learners.  While  it  took  a 
while  for  this  to  sink  in  for  most  learners,  after  the  third  week  of  the  course,  they  had  taken  full  responsibility  for 
those  actions.  As  expected,  each  of  the  four  teams  brought  differing  levels  of  expertise  to  the  situation,  and  as  a 
result,  each  team  was  in  a different  state  of  readiness  as  the  course  moved  along. 

Scott  anticipated  and  observed  a familiar  pattern  to  constructivist  educators  of  initial  resistance,  eventual 
buy-in,  and  intensive  work  culminating  in  a publicly  shared  report.  A pivotal  point  came  around  the  third  week  of 
term,  after  learners  saw  the  scope  and  complexity  of  the  work  they  were  called  to  do,  and  had  to  decide  to  either  buy 
in  or  drop  out.  Some  teams  directly  confronted  Scott  about  the  challenge.  With  assurances  of  support  from  Scott, 
these  teams  resolved  to  continue,  after  which  they  cumulatively  built  a level  of  intensity  in  preparation  for  the  final 
presentation. 

Resolution  is  described  by  Parrish  (2006)  as  “a  feeling  of  unity  or  closure  achieved  when  we  perceive  that 
events  have  reached  resolution  or  an  appropriate  conclusion”  (p.  1).  From  the  outset,  learners  knew  that  they  would 
be  working  in  teams  to  try  to  produce  a winning  bid  for  the  RFP.  As  events  unfolded,  learners  were  informed  that 
the  client  would  host  an  award  banquet  at  the  end  of  the  proposal  period  to  award  one  team  with  the  winning  bid.  At 
some  point  in  the  process,  the  teams  began  discussing  and  looking  forward  to  this  event,  making  it  the  eventual  focal 
point  in  the  process.  Each  team  presented  a final  solution  to  the  client  one  week  prior  to  the  banquet,  but  most  of  the 
learners  considered  the  final  banquet  as  the  culminating  event,  considering  the  feedback  from  the  “client”  and  the 
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facilitator  as  closure  to  the  process.  As  one  learner  described  it,  “I  don’t  think  1 have  ever  been  so  concerned  with  a 
project  after  it’s  actually  been  submitted.” 

Walk-Through  - Mythic  Layer 

Framing  the  management  course  as  an  experiential,  immersive  journey  built  around  a competitive  environment  has  a 
clear  connection  to  Campbell's  (1968)  notion  of  a hero ’s  journey.  By  learners  seeing  themselves  as  protagonists 
engaged  in  a personal  quest  to  emerge  victorious  over  other  participants,  learners  were  directly  involved  in  a process 
of  defining  their  journey  and  engaging  in  an  adventure  that  included  negotiation  and  reconciliation. 

The  RFP  approach  to  the  course  put  the  students  in  peril  - and  that  level  of  risk  and  challenge  is  what  most 
clearly  sets  it  off  as  a hero’s  journey.  Many  courses  are  specifically  designed  to  minimize  peril  - to  make  learners 
feel  completely  safe,  with  incremental  bites  of  content  leading  gently  upward  toward  a satisfying  attainment  at  the 
end.  Not  so  this  course  - its  philosophy  was,  “No  guts,  no  glory!”  In  this  case  the  learners  rose  to  the  challenge,  and 
by  doing  so  earned  a reward  commensurate  with  their  efforts. 

Scott's  role  in  the  course  can  be  construed  in  terms  of  a master  guide.  At  the  beginning  of  the  course,  the 
facilitator  clarified  his  role  as  wearing  two  hats  - as  a representative  of  the  fictional  client,  and  as  a supporter  of 
team  processes.  Learners  were  aware  that  the  entire  project  was  fictitious  in  nature,  of  course.  At  some  point,  they 
would  need  to  suspend  disbelief  about  Scott  and  come  to  see  him  realistically  as  a client  rep.  To  get  information 
about  client  intentions,  learners  had  to  direct  all  questions  and  concerns  to  Scott,  serving  as  client  rep.  However,  in 
an  effort  to  personally  support  each  team,  Scott  also  presented  himself  as  an  additional  team  member  in  each  group. 
While  the  team  could  not  count  on  him  to  produce  products  for  the  RFP,  Scott  made  himself  available,  both  inside 
and  outside  of  class,  to  address  issues,  provide  support,  and  act  as  mentor  whenever  the  teams  asked  for  that  support. 
While  initially,  some  teams  sought  to  use  Scott  in  the  hope  of  getting  inside  information  that  would  give  them  an 
advantage  in  winning  the  bid,  by  the  end  of  the  third  week,  each  team  was  respectful  of  Scott's  dual-hat  and  asked 
for  an  appropriate  level  of  support.  Typically  Scott  acted  as  provocateur  by  presenting  lines  of  inquiry  that 
encouraged  learners  to  think  through  their  concerns  rather  than  provide  easy  answers  to  the  questions. 

In  spite  of  a fairly  low-key  approach  to  mythic  design  (because  Scott  was  not  tuned  into  this  concept  at  the 
time),  learners  picked  up  on  some  themes  themselves.  While  not  required  as  part  of  the  course  syllabus  or  even 
suggested  in  initial  discussions,  once  learners  were  assembled  in  teams,  members  created  their  own  fictitious 
companies,  complete  with  professional  corporate  identity  packages  with  letterhead,  logos,  and  organizational  charts. 
A discernible  process  of  transformation  took  place  as  participants  transitioned  from  their  identities  as  graduate 
students  to  specific  defined  roles  within  their  own  companies.  These  new  identities  were  built  largely  on  practical 
experience,  both  in  and  out  of  the  Master’s  program,  and  allowed  learners  to  exhibit  their  strengths  as  they  engaged 
in  the  journey.  One  learner  in  particular  had  been  compelled  to  take  this  particular  course  out  of  sequence  and  did 
not  have  the  ID  background  of  other  participants  in  the  class.  However,  in  her  professional  life,  she  was  a skilled 
health  care  professional  and  because  the  RFP  was  presented  from  a health  care  provider,  found  a place  for 
contributions  to  the  effort  while  learning  about  managing  instructional  products.  This  further  illustrates  the  need  for 
adaptations  and  improvisational  response,  even  with  carefully  designed  programs. 

Eventually  an  actual  third-party  “client”  was  introduced  to  the  teams  to  add  to  the  authenticity  of  the 
experience.  The  client  was  a highly  experienced  instructional  designer  with  over  20  years  of  experience  in  roles  on 
both  sides  of  this  situation  - as  a client  reviewing  and  accepting  proposals  for  complex  training  projects,  and  as  a 
vendor  preparing  design  plans  to  respond  to  a number  of  RFPs.  Through  his  review,  feedback,  and  focused 
questioning  of  the  teams’  design  decisions,  the  designer  playing  the  role  of  the  client  established  a considerable  level 
of  credibility  among  the  teams,  also  acting  as  a mentor  when  the  situation  surfaced.  The  client  was  present  at  the 
final  banquet  and  actually  did  award  the  bid  - but  took  the  strengths  of  each  proposal  to  propose  that  each  team  be 
awarded  some  section  of  the  contract  in  an  unprecedented  event  for  this  fictitious  company.  Each  team  was 
acknowledged  in  public  for  a particular  unique  strength  and  insight  that  they  brought  to  the  solution.  The  decision 
to  award  a portion  of  the  bid  to  each  of  the  teams  was  based  on  the  quality  of  the  thinking  and  the  quality  of  product 
that  each  team  produced.  It  would  have  been  difficult  in  any  class  circumstance  to  single  one  team  out  when  all  of 
the  teams  worked  very  hard  to  produce  an  exceptional  product,  but  in  this  case,  the  client’s  background  and 
expertise  allowed  him  to  respond  with  this  solution  in  a way  that  maintained  the  credibility  and  integrity  of  the  class 
experience.  While  a couple  of  learners  joked  about  the  nature  of  the  competition , in  the  end,  learners  left  the 
banquet  discussing  the  level  of  closure  that  they  had  sought  and  a level  of  satisfaction  with  the  experience  they 
would  indicate  that  they  did  not  feel  cheated  by  the  final  outcome. 
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Concluding  Thoughts 


The  walk-through  of  the  management  class  is  just  one  example  of  how  the  three-layer  design  for  transformative 
learning  might  be  implemented.  Other  approaches  may  be  more  technology  infusing,  such  as  the  creation  of 
aesthetically  powerful,  mythically  deep  learning  environments  that  draw  on  game-design  elements  (Dickey,  2005, 
2006;  see  also  Davies,  2003).  We  see  the  work  of  Kathy  Sierra  (2006),  intended  to  confront  the  learner  more  directly 
with  an  engaging  and  motivating  learning  experience,  as  largely  consistent  with  the  goal  to  transform  learners  at  a 
deeper  level. 

The  larger  point  of  our  inquiry  rests  on  a theme  touched  on  repeatedly  - that  instructional  design  has  nearly 
reached  a dead  end  in  its  pursuit  of  dramatic  learning  gains  via  instructional  strategy  as  its  primary  mechanism.  The 
next  significant  step  in  advancement,  we  believe,  will  be  in  closer  attention  to  crafting  high-quality  environments 
and  interactions  that  tap  into  the  learner’s  need  for  immerse  or  engaging  experience.  Achieving  really  high-quality 
instruction  a largely  open,  creative  process  that,  like  any  other  design  task,  draws  on  the  creative  and  problem- 
solving resources  available. 

Seeing  ID  as  a careful  crafting  and  controlling  of  elements  - this  brings  it  into  line  with  other  designers  of 
experience  such  as  architects,  human-interface  designers,  industrial  designers,  and  game  designers.  Because  this  area 
has  been  neglected  in  our  field,  we  don’t  know  a lot  about  how  to  do  that.  That  leaves  some  exciting  areas  for 
continuing  inquiry,  such  as: 

• How  do  designers  combine  considerations  of  the  local  problem  space  with  their  repertory  of  strategies  and 
models? 

• How  would  we  recognize  truly  gifted  design  work?  How  we  would  characterize  design  expertise, 
particularly  the  kind  that  leads  to  transformative  learning  experience? 

• What  marks  the  truly  gifted  design  professional  from  his/her  peers?  Are  there  any  approaches,  beliefs,  or 
strategies  that  could  be  useful  to  other  professionals? 

• What  kind  of  professional  rewards,  programs,  or  exchanges  might  serve  the  interests  of  truly  outstanding 
design  work?  Beyond  refereed  journals,  how  could  the  design  community  identify  and  acknowledge 
outstanding  and  creative  design  work  (see  Rowe,  Smith,  & Boling,  2005)? 

• How  can  we  better  understand  the  links  between  designer  intentions,  instructor  intentions  and  activities, 
and  learner  intentions,  activities,  and  constructions? 

To  fully  explore  these  areas  of  inquiry,  we  should  be  able  to  largely  continue  using  present  methods  of 
research  and  inquiry.  Accepted  methods  of  cognitive  analysis  will  continue  to  be  relevant,  broadly  conceived  and 
creatively  applied.  Design  research  should  come  increasingly  into  play.  Proof-of-concept  work  - various  kinds  of 
prototype  designs  or  tool-development  projects  - should  also  be  relevant.  And  similarly  to  architecture  and 
industrial  design,  we  need  to  begin  seeing  exemplary  design  work  itself  as  a form  of  inquiry  and  scholarly 
accomplishment,  complete  with  jury-review  processes  and  acknowledgement  of  excellence. 

The  three-layer  design  approach  asks  designers  to  consider  issues  they  may  not  have  recognized  in  their 
previous  work.  While  we  acknowledge  the  divergent  conceptual  bases  presented  by  the  three  layers,  we  feel  their 
serious  consideration  is  completely  relevant  to  ID  practice.  Effective  ID  models  and  practices  must  draw  on  eclectic 
ideas  and  theories,  and  should  reflect  the  whole  person,  not  just  a narrow  construal  of  behavior  or  cognition  (Wilson, 
2005b;  Merrill  & Wilson,  2007). 

Finally,  we  acknowledge  again  that  transformative  learning  experiences  cannot  be  targeted  and  induced  as 
more  common  learning  goals  can  be.  Even  when  designing  with  all  three  layers  in  mind,  a small  number  of  students 
may  have  a transformative  learning  experience.  Even  so,  we  believe  it  very  healthy  and  productive  to  build  designs 
as  if  they  will  lead  to  transformative  learning.  Working  toward  that  goal  increases  the  likelihood  that  learners  will 
indeed  become  more  deeply  engaged  and  experience  transformative  learning  when  they  are  ready  for  it.  By  raising 
the  standard  and  challenging  designers  to  create  conditions  for  transformative  learning,  we  can  move  from 
technically  satisfactory  instruction  (cf.  Sierra,  2006)  to  the  kind  of  crafted  learning  encounter  that  leaves  an 
impression  and  enters  into  people's  stories  about  themselves  and  their  stance  toward  the  subject  matter. 

Author  Note 

The  IDEAL  Lab  at  the  University  of  Colorado  investigates  Innovative  Designs  of  Environments  for  Adult  Learners; 
see  http://thunderl.cudenver.edu/ideal/.  Additional  lab  members  include:  Nathan  Balasubramanian,  Lee  Christopher, 
Jamie  Hurley,  Lisa  O’Reilly,  and  Bob  Snead. 
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Online  Conferencing  and  Blogs:  Which  do  I use  for  what  and  why? 

David  Winograd  Ph.D. 

Toward  strengthening  connections  between  theory  and  practice,  this  paper  is  concerned 
with  asynchronous  conferencing  and  weblogs  or  blogs.  I will  discuss  the  theoretical  underpinnings  of  both 
mediums,  the  similarities  between  the  two,  how  they  are  currently  being  used  in  schools,  and  offer 
suggestions  for  the  implementation  of  each  medium  for  specific  outcomes  and  contexts  as  well  as 
suggesting  how  both  can  be  used  in  tandem,  each  for  a specific  purpose. 

Computer-mediated  communication  has  been  defined  as  "communication  between  different  parties 
separated  in  space  and/or  time,  mediated  by  interconnected  computers"(Romiszowski  & Mason,  1996). 
The  start  of  CMC  (Computer  Mediated  Communication)  happened  in  the  late  1960’s  when  scientists  using 
the  same  mainframe  computer  found  that  they  could  leave  messages  for  each  other  on  that  computer 
sharing  ideas  and  information.  This  became  the  start  of  email  (Sproull  & Kiesler,  1991).  Since  its 
inception  many  modes  of  CMC  have  been  developed  ranging  from  synchronous  chats  and  instant 
messaging  to  asynchronous  weblogs,  conferences  and  pod  casts.  This  paper  will  discuss  weblogs  and 
computer  conferencing. 

Computer  conferencing  can  be  understood  to  support  many-to-many  communication  by  permitting 
groups  to  communicate  at  times  convenient  to  the  individual  because  of  its  inherent  asynchronicity 
(Feenberg,  1986).  (Mclsaac  & Gunawardena,  1996)  explained  a number  of  features  of  computer 
conferencing  systems,  “In  these  systems,  messages  are  linked  to  form  chains  of  communication,  and  these 
messages  were  stored  on  the  host  computer  until  an  individual  logs  on  to  read  and  reply  to  the  messages” 

(p.  417).  (Eastmond,  1992)  identified  three  components  of  effective  computer  conferencing  in  education: 
first,  there  is  a match  between  or  among  the  medium,  teaching  style,  student  characteristics,  and  course 
content;  second  was  the  overall  course  design;  and  third  was  the  presence  of  a skilled  moderator. 

Weblog  or  blog  is  a term  coined  in  1999  (Catalano,  2005).  The  Wikipedia  online  reference  defines 
a blog  as  a website  in  which  journal  entries  are  posted  on  a regular  basis  and  displayed  in  reverse 
chronological  order.  The  term  blog  is  a shortened  form  of  weblog  or  web  log.  Blogs  are  not  a new  medium, 
and  should  not  be  looked  upon  as  a new  mechanism  as  (Catalano,  2005)  put  it  “Blogs  are  not  the  new 
papyrus”  p.  7. 

There  Are  Many  Similarities  In  The  Structure  Of  Online  Educational  Conferencing  And  Blogs,  Including: 

* Asynchronicity 

* Reflection 

* Ease  of  set  up:  Setting  up  a conference  today  is  most  often  a function  of  the  Course  Management 
Software  that  is  used.  There  are  many  options  for  this,  the  largest  being  BlackBoard,  which  contains  a 
threaded  discussion  space  as  one  of  its  many  features.  Conferencing  in  BlackBoard  is  only  one  of  the 
modules,  while  other  software  such  as  FirstClass,  having  a far  better  and  more  flexible  conferencing 
structure  than  does  BlackBoard,  offers  little  else.  With  the  proper  software,  computer  conferencing  is  easy, 
but  blogging  can’t  be  done  within  the  Course  Management  Software.  FirstClass  provides  only  a subset  of 
the  BlackBoard  feature  set  toward  course  management.  It  does  however,  as  does  Ichat  for  the  Macintosh, 
allow  you  to  drag  and  drop  any  attachments  into  the  conference  window  and  seamlessly  transfer  files. 

A major  reason  that  computer  conferencing  is  used,  is  because  the  school  has  already  bought  the 
software  and  hired  a server  administrator  to  make  sure  things  work.  With  the  administrator  doing  the 
“heavy  lifting”,  bringing  teachers  up  to  speed  on  existing  conferencing  software,  it’s  time  consuming  but 
easy  to  set  up  a conference.  The  concept  of  Course  Management  Systems  and  dedicated  administrators  are 
not  a part  of  the  blogosphere. 

Blog  owners  don’t  worry  about  such  complexity  or  cost  since  blogspace  is  free  or  very  cheap.  If 
you  download  the  software  and  follow  the  instructions,  presto!  You  have  a blog.  Fill  in  the  blanks  and  start 
blogging. 

* Lack  of  the  requirement  of  HTML  skills:  Although  a rudimentary  knowledge  of  HTML  is  quite  useful  in 
course  problem  solving,  today  it  is  far  from  required. 

*Being  easily  publishable:  One  click  publishing  is  standard  for  blogs.  When  conferencing  (and  not  using 
other  modules)  posting  to  the  conference  also  takes  no  more  than  one  click. 
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*A  high  degree  of  maintenance:  This  is  the  reason  that  many  blogs  and  large  computer  conferences  fail. 
Years  ago  I ran  the  Macintosh  Community  Club  Forum  for  CompuServe,  an  early  online  service.  This  was 
an  online  community  of  a few  thousand  people.  Some  days  we  would  get  as  many  as  five  hundred 
messages  per  day.  1 had  to  read  them  all  and  respond  to  many.  As  time  went  on  it  became  bigger  and 
wound  up  taking  me  three  to  five  hours  a day  to  go  through  everything.  Most  people  just  don’t  have  time  to 
run  something  that  big.  I gave  it  up  when  I started  my  doctorate  since  I didn’t  have  the  amount  of  free  time 
I enjoyed  earlier.  A good  blog  is  similar.  It  takes  time  to  write  everyday  or  most  days,  keep  up  with  your 
links,  and  monitor  the  comments  daily.  It’s  more  work  than  first  expected  and  often  gets  old  quickly  after 
people  find  out  what  they  are  in  for. 

*A  high  degree  of  timeliness:  This  can  be  looked  upon  several  ways.  If  a blog  has  anything  to  do  with  news 
or  politics,  it  must  be  revised  just  about  daily.  The  timeliness  of  a computer  conference  relates  to  how  often 
it  is  being  used.  Usually  discussing  a topic  requires  a predetermined  set  of  rules  including:  frequency  of 
posting,  grading  issues,  how  long  the  asynchronous  conference  will  last,  when  to  take  it  down  (Murphy  et 
al.,  1996),  and  who  will  moderate  the  conference. 

Aspects  Attributable  To  Online  Educational  Conferences  But  Seldom  To  Blogs  Include: 

* Specific  topic  orientation:  Instead  of  being  interested  in  a larger  subject,  the  conference  is  most  often 
concerned  with  one  topic. 

* A closed  ended  structure:  When  a subject  is  finished,  the  conference  closes  down. 

* Structural  limitations  to  inclusion  of  multimedia:  In  most  conferencing  software,  such  as  that  found  in 
BlackBoard,  the  only  thing  that  can  be  input  or  uploaded  is  plain  text  or  HTML. 

* An  egalitarian  community  of  equals:  Although  there  is  a moderator  in  successful  conferences,  each 
participant  is  considered  of  equal  value  to  the  debate  or  conversation  at  hand. 

* A trained  moderator  needed  for  success:  A moderator  is  quite  different  from  a blog  owner.  A moderator 
keeps  things  going  while  a blog  owner  provides  the  great  majority  of  content. 

* A degree  of  manipulation:  This  is  not  a bad  thing.  Often  a moderator,  or  even  a participant,  plays  ‘devil’s 
advocate’  to  keep  the  discussion  flowing  and  thereby  instilling  some  degree  of  manipulation. 

* Often  involved  in  problem  solving:  Since  online  discussions  are  closer  to  a debate  than  a journal,  and 
feature  equal  participation,  they  are  often  used  for  brainstorming  or  problem  solving. 

* Structurally  closed:  For  educationally  oriented  conferences,  if  you  are  not  in  the  class,  you  are  not  in  the 
conference. 

* Private  in  nature:  In  most  cases  educational  conferences  are  restricted  to  members  of  the  class  and 
require  a password  or  other  forms  of  authentication. 


Aspects  Attributable  To  Blogs  But  Seldom  To  Online  Conferences  Include: 

* Individual  ownership:  A blog  is  owned  and  run  by  one  individual. 

* Easy  integration  of  multimedia:  Text  is  only  one  type  of  object  that  can  be  uploaded  to  a blog. 

* Being  individual  based:  A great  deal  of  the  success  or  failure  of  a blog  depends  on  the  general 
atmosphere,  and  personality  of  the  blog.  This,  for  good  or  ill,  is  dependent  upon  the  skills  of  the  author. 

* Permanence:  Conceivably,  a blog  can  last  forever  while  a computer  conference  is  almost  always  closed 
down  at  the  end  of  the  semester  or  class. 

* Good  writing  skills  necessary  for  success:  A blog  is  only  as  good  as  the  quality  of  its  owner/author. 

* Theme  but  not  topic  based:  Unlike  a conference,  which  lives  as  long  as  the  class,  blogs  are  based  around 
generally  large  themes  of  interest  while  a conference  most  often  deal  with  specific  topics  within  a theme. 
Different  topics  often  spawn  different  conferences. 

* Structurally  open:  You  can  add  many  more  things  to  a blog  than  with  a computer  conference. 

*Public  in  nature:  A blog  is  open  to  the  public  while  a conference  is  restricted  to  a particular  group  and 
often  requires  authentication  to  access. 

Are  Either  Modalities  Communities  Of  Practice? 

The  term:  community  of  practice  has  been  a buzzword  since  the  early  1990s.  The  term,  often 
shortened  to  CoP,  is  a process  of  social  learning  that  occurs  when  people  with  like  interests  collaborate 
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over  an  extended  period  of  time  to  share  ideas  and  construct  knowledge,  when  Lave  and  Wenger  coined  it 
in  relation  to  situated  learning.  The  concept  of  Communities  of  Practice  were  originally  developed  to 
describe  face  to  face  situations  such  as,  in  its  initial  research,  recovering  alcoholics  in  Alcoholic 
Anonymous,  Meat  Cutters  and  Army  Quartermasters.  It  was  soon  found  that  CoPs  could  exist  well  online, 
which  has  encouraged  a new  strand  of  Computer  Mediated  Communication  (CMC)  research.  According  to 
(Wenger,  n.d.)  a CoP  contains  three  key  elements: 

1.  Domain  of  knowledge:  creates  common  ground 

2.  Community:  creates  the  social  fabric  of  learning 

3.  Practice:  a set  of  frameworks,  ideas  and  tools  that  community  members  shared. 

This  would  seem  to  suggest  that  both  online  conferences  and  blogs  could  easily  be  considered 
CoPs.  However  in  1998,  Wenger  extended  the  concept  to  include  organizations  and  provided  dimensions 
for  differing  types.  In  education,  he  proposed,  there  are  three  dimensions  that  need  obtain  to  bring  the 
experience  of  schooling  to  everyday  life: 

Internally:  How  to  ground  school  learning  experiences  in  practice  through  participation  in  communities 
around  subject  matters. 

Externally:  How  to  connect  the  experience  of  students  to  actual  practice  through  peripheral  forms  of 
participation  in  broader  communities  beyond  the  walls  of  the  school. 

Over  the  lifetime  of  students:  How  to  serve  the  lifelong  learning  needs  of  students  by  organizing 
communities  of  practice  focused  on  topics  of  continuing  interest  to  students  beyond  the  schooling  period 
(Wenger,  2001). 

The  third  required  dimension  does  not  hold  with  educational  computer  conferences  since  when  a 
class  ends,  so  does  the  conference,  which  is  typically  kept  on  a University  server  and  cleared  to  make  room 
for  next  semester.  A blog,  however,  does  not  always  reside  on  a school  server  so  it  can  go  on  as  long  as  its 
creator  would  like  it  to  survive.  This  is  in  tune  with  lifelong  learning  since  the  same  blog  can  be  used  for 
many  years  of  classes  with  each  class  enriching  its  information  and  utility.  Therefore  a computer 
conference  cannot  be  a CoP  while  a blog  can. 

When  To  Use  Computer  Conferencing 

Computer  conferencing  is  an  excellent  way  to  discuss  and  debate  specific  issues,  or  themes.  They 
are  designed  to  allow  members  of  the  class  to  interact  online  in  a moderator  (often  the  teacher  is  the 
moderator)  driven  space.  Unlike  blogs,  computer  conferencing  requires  some  form  of  Course  Management 
Software,  like  BlackBoard,  which  is  expensive  but  required  to  start  a closed  conference,  while  a blog  can 
be  run  on  software  that  is  either  extremely  cheap  or  free.  Although  it  is  possible  to  run  a computer 
conference  without  a Course  Management  System  by  finding  a free  system  on  the  Internet,  these  are 
difficult  to  set  up  and  look  quite  amateurish  Since  having  someone  post  something,  someone  else  coming 
back  later  and  responding  to  that  message,  is  geared  toward  debate,  online  debates  make  for  a great  use  of 
computer  conferencing. 

Computer  conferencing  is  a good  way  to  instill  critical  thinking  into  a curriculum.  Often  a 
classroom  doesn’t  have  time  to  spend  on  a macro  view  of  a field  of  study.  So  for  distance  learning, 
classroom  learning,  and  hybrid  learning  it’s  useful  to  have  an  online  asynchronous  discussion,  based  on 
information  relevant  to  the  class.  It’s  also  useful  for  the  teacher  who  not  only  gets  a better  idea  of  how  the 
students  in  her  class  think,  but  an  idea  of  the  concepts  that  are  hard  for  the  class  to  grasp.  Once  this  is 
determined,  the  next  conference  can  be  built  around  what  needs  most  work.  If  computer  conferencing  is 
used  group  size  must  be  small.  It  has  been  found  that  a good  size  for  a conference  is  between  12-15  people 
(Murphy  et  ah,  1996).  If  the  group  is  too  small,  people  will  become  self-conscious  and  feel  like  they  are 
always  ‘on’.  If  a group  is  too  large,  people  get  lost  in  the  textual  shuffle  and  if  a requirement,  such  as  three 
messages  per  week,  is  instituted  the  amount  of  messages  to  be  read  becomes  unwieldy  for  both  the 
students  and  the  moderator  or  teacher.  Large  groups  also  make  it  hard  to  debate  since  a common  complaint 
is  that  a student  wanted  to  post  a new  idea  but  someone  beat  her  to  it. 

In  poorly  designed  computer  conferences  there  is  rampant  case  of  “me-too-ism.  This  is  when 
someone  writes  a thoughtful  message  with  which  another  person  agrees.  That  someone  else  might  copy  the 
entire  multipage  message  into  his  clipboard  and  then  post  a message  with  the  entire  message  copied  to  the 
new  message  and  the  only  input  that  someone  else  has  written  is  something  like  “I  agree”.  I’m  sure  that 
you  can  see  that  group  members  would  be  disgruntled  over  such  a time  waster.  So  it  is  up  to  the  teacher  to 
set  the  ground  rules.  A few  of  these  that  have  proven  successful  are  to  limit  messages  to  only  new  ideas  that 
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haven’t  already  been  posted,  setting  a no  less  than  a certain  number  of  messages  per  week,  and  having 
messages  contain  at  least  two  paragraphs. 


When  To  Use  Blogs 

Blogs  are  journalistic  in  nature.  Journals  are  a common  strategy  to  aid  students  with  metacognition 
hoping  that  students  will  not  only  learn  better  but  also  learn  more  about  themselves.  Papers  assigned  by 
teachers  are  read  by  an  audience  of  one  and  have  a propensity  of  saying  what  the  teacher  wants  to  hear 
(Hernandez-Ramos,  2004).  But  by  using  a blog  students  often  spend  more  time  writing  since  along  with  the 
teacher,  others  both  inside  and  outside  the  class  can  read  it  offering  it  up  to  an  unknown  audience  (Levin  & 
Camp,  2002).  This  also  makes  blogs  more  reflective  than  conferences,  which  are  most  often  used  for 
debating  specific  topics  in  a hit  and  run  nature  and  need  a trained  moderator  to  keep  the  discussion  flowing 
(Winograd,  2000).  The  vast  majority  of  blogs  are  individually  owned.  It  is  in  the  skills,  perseverance,  and 
enthusiasm  of  the  owner  that  contributes  strongly  to  the  success  or  failure  of  the  blog. 

The  concept  of  the  blog,  in  retrospect,  was  inevitable  since  in  the  discussion  boards  on 
CompuServe  it  was  found  that  a tiny  fraction  of  the  people  reading  the  discussions  actually  posted 
anything.  They  liked  to  read  the  prose  of  group  of  about  50  people  who  posted  quite  often.  This  is  not  the 
case  with  a blog,  where  the  owner  supplies  most  of  the  text.  Blogs,  and  let’s  limit  this  to  educational  blogs, 
are  created  by  the  teacher  and  are  often  used  in  English  Composition  classes  and  other  writing  intensive 
courses  for  a start. 

Ask  a student  if  he  or  she  often  reads  and  writes  outside  of  class.  Chances  are  that  they  don’t.  But 
if  you  ask  them  if  they  use  Instant  Messenger  and  Email,  you  will  receive  a far  warmer  response.  Often, 
what  children  learn  through  email  and  Instant  Messenger  can  be  transferred  to  blogs,  which  is  a digital 
piece  of  paper  to  write  upon.  There  are  a number  of  reasons  to  have  classroom  blogs  in  the  liberal  arts. 

What’s  In  It  For  A Teacher  To  Run  A Class  Blog? 

There  are  a number  of  things  according  to  Jay  Cross  in  an  educational  blog: 

• Weblogs  are  a personal  writing  space,  easy,  sharable  and  automatically  archived. 

• Weblogs  are  easily  linked  and  cross-linked  to  form  learning  communities. 

• Weblogs  can  become  a digital  portfolio  of  students’  assignments  and  achievements. 

• Weblogs  are  a novice’s  web  authoring  tool. 

• Via  digital  storytelling,  weblogs  play  a role  in  personal  development. 

• Accumulated  Weblogs  become  a content  management  system.  (Elliot  et  al,  2006) 

The  only  part  of  this  analysis  is  that  I question  is  if  an  accumulation  of  blogs  can  be  considered  a 

Course  Management  System  since  I consider  a CMS  containing  a highly  sophisticated  suite  of 

features. 

Anne  Davis  has  some  slightly  more  generalized  reasons.  She  sees  blogs  as: 

• A way  to  improve  my  writing. 

• A just  in  time  learning  arena. 

• A place  to  share. 

• An  easy  way  to  create  a website 

• A way  to  connect  with  others  and  make  connections  to  learning. 

• A perfect  spot  for  quick  writes. 

• A way  to  take  ownership  of  a personal  space. 

• A writing  room  in  which  you  can  make  it  be  what  you  want  it  to  be. 

• A place  to  go  each  day  that  provides  elements  of  surprise  and  anticipation. 

• A way  to  give  students  ownership  of  a personal  space  that  encourages  active  engagement  by  the 

students  and  teacher. 

• A place  to  collaborate 

• An  up  close  and  personal  way  to  include  parents  in  the  process  (Elliot  et  al.,  2006) 

The  last  point  is  particularly  salient.  Blogs  can  be  used  to  communicate  with  parents.  As  the 
student  puts  together  a digital  portfolio,  his  parents  can  check  it  on  the  class  blog,  which  is  not  password 
protected  or  authenticated.  Unlike  computer  conferences,  blogs  are  open  to  anyone  with  an  Internet 
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connection.  Since  a blog  can  be  used  as  a digital  portfolio  both  teacher  and  parents  can  keep  abreast  of  a 
student’s  work. 

Blogs,  but  not  computer  conferences,  can  last  longer  than  the  semester.  That  is  usually  a very  good 
thing  since  over  time  a blog  can  become  a repository  of  knowledge  and  the  longer  it  runs,  the  deeper 
becomes  its  content.  It  can  have  permanence.  Over  time,  a blog  can  become  a database  of  knowledge  based 
upon  years  of  information  students  post  on  either  the  class,  or  their  personal  blog. 

Educators  are  using  blogs  in  all  fields,  but  I consider  it  most  appropriate  for  liberal  arts  and 
especially  creative  writing  classes.  In  this  case  all  assignments  can  be  submitted  via  the  blog  either  by 
uploading  it  or  by  posting  it  to  their  own  student  blog.  A most  interesting  use  of  educational  blogging  is 
Exploring  Techno/Literacy  (Fornes,  2003).  In  this  class,  Fornes  created  a blog  to  serve  as  a central  hub  and 
then  had  each  of  his  students  start  their  own  blog,  which  were  linked  to  the  main  class  blog.  Also  to  be 
found  on  the  main  blog  page  are  links  to  the  syllabus,  library  and  other  classroom  information.  In  effect  this 
blog  serves  the  same  purpose  as  putting  up  a class  webpage,  but  it’s  easier  to  create  and  interactive.  Click 
on  a student’s  name  and  section  and  you  will  be  sent  to  that  student’s  personal  blog.  In  this  case,  the  blog 
was  used  for  submission  of  creative  writing  assignments  and  not  much  else.  At  the  end  of  a semester  the 
students  have  a digital  portfolio  of  their  work. 

Blogs,  unlike  computer  conferences,  present  the  student  with  a blank  sheet  of  electronic  paper  that 
can  be  used  to  journal,  submit  assignments,  or  just  write  about  what  the  student  feels  is  important. 
Obviously  educationally  oriented  blogs  created  and  run  by  students  are  more  concerned  with  specific  topics 
than  are  non-educational  personal  blogs,  which  can  be  used  for  just  about  anything. 

Blogs  create  a personal  space,  owned  by  the  author  while  computer  conferencing  is  a communal 
space  owned  by  the  teacher  or  school.  Continuing  Fornes  example,  an  author  can  post  an  assignment, 
which  will  only  be  read  by  someone  accessing  the  individual  blog.  This  provides  more  of  a feeling  of 
freedom  and  ownership  than  does  the  question  and  answer  format  of  a computer  conference,  where 
everyone  is  supposed  to  read  everything  before  responding.  Blogs  often  need  no  response  and  in  reading 
many  blogs,  people  commenting  on  a blog  entry  are  few  and  far  between.  Therefore  blogs  can  be  seen  as  a 
way  to  personally  publish,  while  computer  conferences  are  a way  to  discuss. 

When  To  Use  Both  Blogs  And  Computer  Conferencing 

Using  blogs  and  computer  conferencing  in  a class  is  not  mutually  exclusive.  Both  serve  different 
purposes  and  are  used  in  different  ways.  Assuming  that  a school  already  has  a Course  Management  System 
in  place,  a teacher  can  consider  blogs  and  computer  conferencing  as  two  different  tools  to  wear  on  their 
educational  tool  belt.  Each  teacher  must  decide  what  outcomes  are  desired  and  use  the  correct  tool  or  tools 
to  affect  that  outcome.  Although  it  may  sound  confusing  to  use  both  conferencing  and  blogs  in  one  class,  it 
makes  sense.  Debates  of  issues  by  equal  participants  are  well  served  by  computer  conferencing.  Free  or 
creative  writing,  assignment  submission  where  it  is  not  expected  that  everyone  in  class  read  everyone’s 
entries  are  fodder  for  blogs.  In  the  case  of  some  course  blogs,  one  of  the  links  on  the  top  of  the  screen  is 
‘BlackBoard’.  This  makes  it  quite  easy  to  go  from  a blog  page  to  a computer  conference  by  simply 
clicking  the  link. 

Using  both  together  adds  a richness  that  can’t  be  found  in  either  one.  In  fact  using  them  in  tandem 
provides  some  interesting  possibilities.  An  example  of  this  would  be  that  someone  writes  something 
provocative  on  his  or  her  personal  blog.  The  teacher  or  moderator  decides  that  the  message  would  be  a 
good  topic  for  discussion,  so  the  teacher  or  moderator  starts  a computer  conference  discussing  and  debating 
the  blog  entry.  Alternatively,  when  a computer  conference  is  shut  down  since  everyone  has  made  their 
opinions  known,  a blog  entry  can  be  assigned  to  synthesize  the  discussion  and  show  how  each  individual, 
having  read  all  the  messages,  has  come  to  a set  of  conclusions,  which  will  be  written  to  their  blog.  This  can 
be  quite  useful  since  in  computer  conferencing  the  responses  are  usually  much  shorter  than  blog  entries,  so, 
students  writing  blog  entries  in  places  of  pseudo-privacy  are  free  to  expand  upon  their  conclusions.  This  is 
helpful  since  in  computer  conferencing,  opinions  are  stated  and  sometimes  even  backed  up,  but  not  many 
people  would  remember  each  class  member’s  position  on  the  topic. 

Above  are  some  basic  examples,  but  the  best  parts  of  using  computer  conferencing  with  blogs  are 
yet  to  be  discovered.  Although  blogs  have  been  around  for  a number  of  years  and  they  have  been  used  in  a 
myriad  of  different  ways,  no  one,  to  my  knowledge,  has  found  the  perfect  symbiosis  of  both  media.  It’s  up 
to  the  teachers  to  find  ways  to  make  best  use  of  the  tools  for  his  or  her  purposes.  There  is  a huge  body  of 
literature  relating  to  computer  conferencing  starting  in  the  early  nineties  (Eastmond,  1992),  but  research 
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into  blogs  is  at  a very  early  stage.  Hence,  many  teachers  are  using  blogs  by  trial  and  error.  There  is  an 
increase  in  teachers  using  blogs,  so  there  will  be  more  experimentation  in  blog  use  and  its  tie-in  to 
computer  conferencing.  As  time  goes  by  I expect  to  see  some  exciting  uses  of  the  media  by  combining  the 
two. 
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Abstract 

This  paper  presents  how  a sound  practical  approach  incorporating  with  teaching  presence  has  been 
designed  and  implemented  in  a graduate  asynchronous  on-line  course,  including:  the  scaffolding  before  starting  new 
learning  topics;  the  transaction  during  the  learning  process;  the  evaluation  during  and  after  each  learning  topic;  and 
the  presentation  of  outcomes  online  (STEP).  The  effectiveness  of  STEP  on  students’  social  presence  is  examined 
after  the  course. 

Keywords:  Social  Presence,  Teaching  Presence,  Online  Instructional  Modeling 

Introduction 

The  term  of  social  presence  was  first  introduced  by  Short,  Williams,  and  Christie  (1976),  which  has  been 
defined  as  the  “degree  of  salience  of  the  other  in  a mediated  communication  and  the  consequent  salience  of  their 
interpersonal  interactions”  (p.  65).  They  regard  social  presence  as  “it  varies  among  different  media,  it  affects  the 
nature  of  the  interaction  and  it  interacts  with  the  purpose  of  the  interaction  to  influence  the  medium  chosen  by  the 
individual  who  wishes  to  communicate”.  Being  in  an  early  period  of  investigation,  studies  which  represented  by 
Short,  Williams,  and  Christie  emphasized  social  presence  as  a quality  of  the  communication  medium  itself,  thus  the 
inability  of  a variety  of  media  to  transmit  nonverbal  cues  would  affect  interpersonal  communication  (Tammelin, 
1998;  Rourke,  Anderson,  Garrison,  and  Archer,  2001).  These  “filtered-cues”  studies  postulate  that  users  of 
communication  media  are  in  some  sense  aware  of  the  degree  of  social  presence  of  each  medium  and  therefore  tend 
to  avoid  using  the  particular  medium  for  certain  types  of  interactions  (Tammelin,  1998;  Richardson  and  Swan, 
2003).  Gunawardena  and  Zittle  (1997),  on  the  other  hand,  found  that  social  presence  can  “be  cultured”  among 
teleconference  users,  “a  position  different  from  the  view  that  social  presence  is  largely  an  attribute  of  the 
communication  medium”  (Richardson  and  Swan,  2003,  p.  70).  Gunawardena  and  Zittle  (1997)  argued  that  “in 
reviewing  social  presence  research,  it  is  important  to  examine  whether  the  actual  characteristics  of  the  media  are  the 
causal  determinants  of  communication  differences  or  whether  users’  perceptions  of  media  alter  their  behavior”. 

Furthermore,  Garrison,  Anderson,  and  Archer  (2000)  developed  the  model  of  Community  of  Inquiry  which 
identified  three  essential  elements  for  a successful  higher  educational  experience  on  online  teaching  and  learning, 
particularly  in  the  text-based  asynchronous  environment:  cognitive  presence,  social  presence,  and  teaching  presence. 
In  this  model,  cognitive  presence  has  been  defined  as  “the  extent  to  which  the  participants  in  any  particular 
configuration  of  a community  of  inquiry  are  able  to  construct  meaning  through  sustained  communication”  (p.  89). 
Social  presence  has  been  defined  as  “the  ability  of  participants  in  the  Community  of  Inquiry  to  project  their  personal 
characteristics  into  the  community,  thereby  presenting  themselves  to  the  other  participants  as  ‘real  people’”  (p.  89). 
They  claimed,  “appropriate  cognitive  and  social  presence,  and  ultimately,  the  establishment  of  a critical  community 
of  inquiry,  is  dependent  upon  the  presence  of  a teacher”.  In  one  of  their  further  studies,  Anderson,  Rourke,  Garrison, 
and  Archer  (2001)  defined  teaching  presence  as  “the  design,  facilitation,  and  direction  of  cognitive  and  social 
processes  for  the  purpose  of  realizing  personally  meaningful  and  educationally  worthwhile  learning  outcomes”  (p. 

5).  They  classified  teaching  presence  into  three  categories  - design  and  organization,  facilitating  discourse  and  direct 
instruction,  and  provided  indicators  for  each  category. 

This  paper  presents  how  a sound  practical  approach  incorporating  with  teaching  presence  has  been 
designed  and  implemented  in  a graduate  asynchronous  online  course,  including:  the  scaffolding  before  starting  new 
learning  topics;  the  transaction  during  the  learning  process;  the  evaluation  during  and  after  each  learning  topic;  and 
the  presentation  of  outcomes  online  (STEP).  The  effectiveness  of  STEP  on  students’  social  presence  is  examined 
after  the  course. 
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STEP  Approach  via  Teaching  Presence 


The  course,  entitled  Multimedia  and  Internet  for  Educators,  is  an  elective  graduate  course  from  the  State 
University  of  New  York  at  Oswego.  It  provides  an  introduction  and  guide  to  pre-  and  in-service  teachers  to  current 
and  emerging  technology.  It  is  intended  to  help  PK-12  educators  not  only  to  use  multimedia  and  Internet  resources 
in  their  own  education  but  also  to  integrate  them  into  their  work  as  teachers.  In  spring  200 1 , this  course  was  joined 
State  University  of  New  York  Learning  Network  (SLN)  as  one  of  asynchronous  learning  network  (ALN)  courses 
offering  for  both  on  and  off  campus  students.  Like  most  ALN  courses,  the  Course  Map  of  this  course  includes  three 
areas:  Course  Information,  Class  Community,  and  Learning  Modules  (see  Figure  1). 

Figure  1:  ALN  Course  Map 
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Strategies  of  promoting  active  learning  community  and  enhancing  social  presence  for  this  online  course 
have  emerged  as  a result  of  using  a systematic  approach  to  break  the  isolation  between  students  and  the  instructor, 
students  and  students,  and  students  and  their  learning  environment;  and  to  ensure  the  meaningfulness  of  Web-based 
communication.  Figure  2 illustrates  STEP  is  essentially  a cyclic  process  and  the  four  components  of  STEP  are 
interrelated. 


Figure  2:  STEP  Approach 
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The  S in  STEP 


The  S in  STEP  stands  for  the  scaffolding  before  starting  new  learning  modules.  The  basic  idea  of 
scaffolding  is  “to  gradually  ease  students  into  what  are  likely  to  be  challenging  tasks  by  creating  a supportive 
structure  to  guide  their  work.  In  other  words,  as  the  educators  we  would  initially  do  some  of  the  work  for  students” 
(Grabe  and  Grabe,  1998,  p.  217).  A large  amount  of  students  participated  in  the  course  are  either  with  little 
experience  in  asynchronous  distance  learning  or  with  limited  skills  and  understandings  on  this  type  of  learning 
which  has  multiple  threads  with  several  discussions  and  interactions  progressing  simultaneously.  To  reduce  the 
community  anxiety  among  students,  the  instructor  has  constructed  an  open  discussion  on  the  bulletin  board  by 
sharing  his  own  experience  on  distance  learning  and  background  on  computer  technology,  and  has  invited  students 
to  participate  in  this  activity.  Since  this  activity  is  hosted  at  the  beginning  of  learning  modules,  it  helps  in 
establishing  a non-threatening  atmosphere.  On  each  “Mini-Lecture”  in  learning  modules,  the  instructor  has  started 
with  the  related  concepts/knowledge  and  particular  technological  skills  that  students  need  to  prepare  their 
undertaking  actual  assignments/projects;  provided  resources  links  and  examples;  and  encouraged  students  to  reflect 
and  discuss  the  possibilities  for  extending  the  ideas  and  technological  skills  into  real  world  situations.  Overall,  the 
learning  sequence  of  this  course  includes  three  individual  and  group  projects,  with  each  project  level  of  difficulty  a 
little  higher  than  the  previous  one  and  with  each  project  building  on  the  one  before,  they  are:  1)  using  search  engine 
and  Web  evaluation  criteria  to  organize  their  own  webliography,  in  which  selected  and  listed  Internet  resources  can 
be  used  for  the  future  references  or  projects;  2)  building  on  an  inquiry-oriented  WebQuest  activity,  in  which  some  or 
all  of  the  information  that  learners  interact  with  come  from  resources  on  the  Internet  (Dodge,  1997);  and  3) 
developing  a Web-based  portfolio,  in  which  students  engage  in  continuous,  thoughtful  analysis  of  their  learning  on 
this  course  with  reflection,  evidence,  and  collaboration. 

As  Table  1 shows,  this  component  encloses  categories  and  indicators  of  teaching  presence,  which 
Anderson,  Rourke,  Garrison,  and  Archer  (2001)  have  defined. 

Table  1 

Scaffolding  via  Teaching  Presence 


Categories 

Indicators 

Design  and  organization 

Setting  curriculum 
Designing  methods 

Facilitating  discourse 

Setting  climate  for  learning 

Drawing  in  participants,  prompting  discussion 

Direct  instruction 

Presenting  content  / questions 
Injecting  knowledge  from  diverse  sources 

The  T in  STEP 

The  T in  STEP  stands  for  the  transaction  during  the  learning  process.  ALN  brings  some  new  challenges  in 
practice:  on  one  hand,  students  have  to  deal  with  the  lack  of  personal  touches,  no-verbal  cues,  and  delayed 
responses;  on  the  other  hand,  instructor  may  carry  off  the  real  discussions  among  students  if  he  or  she  merely  acts  as 
“sage  on  the  stage”  (Yang  and  Maina,  2004).  Moore  (1972)  found  that  distance  education  was  characterized  by  the 
amount  of  control  exercised  by  the  learner  (dialogue)  and  the  amount  of  control  exercised  by  the  instructor 
(structure).  “Additional  structure  tends  to  increase  distance  (decreases  community),  and  more  dialogue  tends  to 
decrease  distance  (increase  community)”  (Rovai,  2002,  p.  44).  To  meet  these  challenges  and  to  keep  students 
actively  participating  in  learning  activities,  the  instructor  of  this  course  has  used  “Private  Folder”  serving  as  the 
“On-line  Office  Hours”  to  promptly  respond  to  students’  questions,  and  to  privately  acknowledge  those  students 
whose  ideas  have  been  posted  without  many  or  any  responses  and  feedback.  Meanwhile,  when  common 
questions/concerns  appeared,  the  instructor  has  interacted  with  students  by  posting  his  thoughts  on  the  bulletin  board 
or  discussion  area.  Below  is  an  example  of  a message  from  the  instructor  that  has  been  used  in  early  stage  of  the 
course  to  encourage  students  on  establishing  a socially  responsive  discourse  with  the  course  discussion  (see  Figure 

3). 


499 


Figure  3:  A Message  from  the  Instructor 
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BULLETIN  BOARD  Document  by:  Harrison  Yang  (05/31  10:36  PM) 
Subject:  Sharing  with  the  class 


I truly  believe  that  we  have  a great  start  on  our  class.  So  far,  I enjoy  what  we  are  doing,  and  learning  a lot  from  all  of 
you.  Even  though  I don't  discuss  quite  often  among  our  discussion  (and  plan  to  do  this  later  on),  I do  read  and  think 
your  comments  everyday.  The  reason  I would  use  "On-line  office  hours"  (through  your  private  folder)  to  touch  with 
you  individually  instead  of  jumping  into  discussion  often  -- 1 don't  want  to  set  any  limitations  or  directions  for  our 
reactions.  So,  for  those  who  already  started  to  discuss  the  issues  of  this  module,  let's  keep  up  our  good  work!  For 
those  who  somehow  are  still  "very  quiet"  there,  please  start  to  say  something  - I know  it's  a little  bit  strange  by 
using  keyboard  to  communicate  among  us,  but  please  feel  free  to  do  so.  What  do  we  lose  anyway?  We  are  all  in  the 
same  boat  now:) 

Harrison 


As  Table  2 shows,  this  component  encloses  categories  and  indicators  of  teaching  presence,  which 
Anderson,  Rourke,  Garrison,  and  Archer  (2001)  have  defined. 

Table  2 

Transaction  via  Teaching  Presence 


Categories 

Indicators 

Design  and  organization 

Establishing  time  parameters 
Utilizing  medium  effectively 

Facilitating  discourse 

Seeking  to  reach  consensus  / understanding 
Encouraging,  acknowledging,  or  reinforcing 
student  contributions 

Direct  instruction 

Focusing  the  discussion  on  specific  issues 
Diagnosing  misconceptions 
Responding  to  technical  concerns 

The  E in  STEP 

The  E in  STEP  stands  for  the  evaluation  during  and  after  each  learning  module.  Periodically  evaluating 
students’  performances  serves  two  ends:  to  remind  inactive  students  to  contribute  ideas  and/or  react  to  others’ 
contributions;  and  to  reinforce  interactive  students  continuing  their  journey  on  their  knowledge  and  skills  from 
emergent  to  mastery  (Jonassen,  2000).  Previous  research  of  teacher  immediacy  has  found  that  having  a high  regard 
for  students  work,  actions,  or  comments  can  contribute  to  teacher  immediacy  and  subsequently  to  affective, 
behavioral,  and  cognitive  learning  (Gorham,  1988;  Gorham  and  Zakahi,  1990).  Further,  Rourke,  Anderson, 

Garrison,  and  Archer  (2001)  pointed,  “reinforcement  is  the  object  that  fuels  the  development  and  maintenance  of 
interpersonal  interaction.  Complimenting  and  acknowledging,  and  expressing  appreciation  are  ways  of 
communicating  reinforcement  in  a text-based  medium”.  From  this  perspective,  the  instructor  of  this  ALN  course  has 
privately  sent  every  student  a brief  evaluation  report  every  other  week  to  comment  the  frequency  and  gravity  of  her 
or  his  postings.  Meanwhile,  the  instructor  has  publicly  encouraged  students  to  periodically  summarize  the  discussion 
to  avoid  it  becoming  too  fragmented  or  drifting  off  the  theme. 

As  Table  3 shows,  this  component  encloses  categories  and  indicators  of  teaching  presence,  which 
Anderson,  Rourke,  Garrison,  and  Archer  (2001)  have  defined. 

Table  3 

Evaluation  via  Teaching  Presence 


Categories 

Indicators 

Design  and  organization 

Establishing  time  parameters 
Utilizing  medium  effectively 
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Facilitating  discourse 

Identifying  areas  of  agreement  / disagreement 
Encouraging,  acknowledging,  or  reinforcing 
student  contributions 
Assessing  the  efficacy  of  the  process 

Direct  instruction 

Summarizing  the  discussion 

Confirming  understanding  through  assessment 

and  explanatory  feedback 

The  P in  STEP 

The  P in  STEP  stands  for  the  presentation  on  outcomes.  There  are  a variety  of  ways  to  launch  active,  reflective 
learning.  Providing  opportunities  for  students  to  present  their  ideas,  methods,  and  products  is  one  of  the  most 
powerful  ways.  This  is  essential  not  only  at  the  conclusion  of  a project,  but  also  as  the  project  grows.  Presenting 
projects  is  an  authentic  activity  that  provides  an  enormous  motivation  for  students  (Wheatley,  1991;  Grabe  and 
Grabe,  1998;  Yang,  2001).  “Presentations,  coupled  with  authentic  outcomes  and  fairly  explicit  criteria  for  what 
counts  as  a good  plan,  can  provide  a strong  incentive  to  prepare  and  revise”  (Barron  and  the  Cognition  and 
Technology  Group  at  Vanderbilt,  1998,  p.  286).  Due  to  the  nature  of  asynchronous  learning,  instead  of  the  face-to- 
face  presentation,  students’  projects  have  been  opened  on  the  online  course  periodically  with  the  permission  from 
students.  Through  this  activity,  students  could  share  and  compare  their  projects  with  projects  generated  by  their 
classmates,  which  might  inspire  them  to  revise  and  communicate  further  more. 

As  Table  4 shows,  this  component  encloses  categories  and  indicators  of  teaching  presence,  which 
Anderson,  Rourke,  Garrison,  and  Archer  (2001)  have  defined. 

Table  4 

Presentation  via  Teaching  Presence 


Categories 

Indicators 

Design  and  organization 

Establishing  time  parameters 
Establishing  netiquette 

Facilitating  discourse 

Encouraging,  acknowledging,  or  reinforcing 
student  contributions 

Drawing  in  participants,  prompting  discussion 

Direct  instruction 

Focusing  the  discussion  on  specific  issues 
Injecting  knowledge  from  diverse  sources 

STEP  on  Social  Presence 

To  assess  the  effectiveness  of  the  STEP  approach  on  social  presence  for  online  collaborative  learning,  the 
final  self-reports  of  students  from  spring  2006  (n  = 23)  and  summer  2006  (n  = 25)  were  collected  and  examined, 
which  were  based  on  their  own  experiences  on  the  course  Multimedia  and  Internet  for  Educators.  Findings  from 
students’  final  written  reports  reflected  positive  student  reactions  in  the  effectiveness  of  utilizing  the  STEP  approach 
on  the  course  Multimedia  and  Internet  for  Educators.  Student  responses  related  to  use  of  the  STEP  approach  for 
asynchronous  online  collaborative  learning  were  extremely  encouraging  on  the  aspects  of  social  presence,  which  has 
be  classified  into  three  categories  by  Garrison,  Anderson,  and  Archer  (2000)  - emotional  expression,  open 
communication,  and  group  cohesion,  as  was  evident  in  the  following  students’  comments. 

All  who  participated  were  extremely  courteous,  professional,  helpful,  understanding,  and  tolerant. 
Discussions  were  in-depth  and  thought-provoking.  It  was  almost  hard  to  be  “silent”  even  if  one  wanted  to 
since  there  was  such  diversity  in  the  discussions.  Any  time  I posted  a question  or  needed  clarification  of 
some  sort  there  was  always  someone  who  would  respond  - and  most  of  the  time  I got  more  than  one 
response.  I also  enjoyed  the  sharing  of  real  life  experiences  by  fellow  students,  most  of  whom  are  teachers. 
It’s  great  to  hear  firsthand  accounts  about  what  different  schools  are  doing  with  respect  to  new 
technologies.  I was  introduced  to  many  new  concepts  because  of  it. 
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I have  taken  one  other  online  course.  It  was  a general  education  requirement  for  my  undergraduate  degree. 
By  comparing  the  two  courses,  I can  easily  tell  that  this  course  is  at  the  graduate  level  and  that  the 
participants  are  “experts”  in  their  field.  The  discussion  topics  and  responses  that  were  posted  throughout 
the  duration  of  this  course  were  thorough  and  well  thought  out.  The  issues  and  topics  that  my  classmates 
raised  forced  me  to  think  critically. 

An  interesting  spin  was  to  have  group  work  assignments.  I did  not  expect  group  work  in  an  online  class. 

At  first  I was  a little  apprehensive  of  the  online  groups,  but  despite  my  skeptic  attitude  in  the  beginning  the 
group  was  very  productive.  The  computer  has  become  such  a large  part  of  our  communication  in  everyday 
life;  it  was  almost  like  second  nature. 

This  type  of  setting  allows  some  additional  time  needed  to  foster  a more  thoughtful  and  developed 
response.  This  response,  once  formulated,  can  then  be  expressed  in  the  manner  that  the  creator  had 
envisioned.  If  anything,  I believe  this  course  to  be  more  inclusive.  For  better  or  worse,  human  beings  have 
many  biases  and  preconceptions  about  others.  The  anonymous  nature  of  this  course  effectively  removes 
many  of  the  shallow  biases  and  prejudices  that  exist. 

I have  time  to  formulate  a response  to  my  classmates’  questions  without  feeling  “on  the  spot”  during  that 
awkward  moment  of  silence,  with  all  eyes  fixed  upon  me. 

Through  my  distance  learning  experience,  1 have  had  the  opportunity  to  interact  with  my  peers  located  all 
over  New  York  State.  Together,  we  created  a learning  environment  that  was  comfortable  and  appreciative 
of  others  thoughts,  ideas  and  experiences.  The  discussions  are  very  appealing  to  a new  teacher.  The 
comments  made  by  teachers  with  their  own  classrooms  help  out  the  teachers  that  are  not  in  that  position 
just  yet.  Some  comments  made  push  our  thoughts  to  be  out  of  the  box  at  times,  I think  that  reflects  our 
comfort  level  with  our  classmates  and  the  professor. 

Before  this  course  I must  say  I hadn’t  explored  WebQuests  in  depth.  I am  so  excited  to  have  the 
WebQuests  my  classmates  created.  I feel  that  they  will  be  a valuable  resource  in  the  future.  Exploring  their 
WebQuests  really  opened  my  eyes  to  the  amount  of  possibilities  which  lie  in  using  them  in  the  classroom. 

I was  also  anxious  to  see  my  peer’s  final  projects.  I really  liked  that  we  could  see  each  others’  projects.  I 
printed  off  some  of  my  peers’  work  that  1 thought  might  be  beneficial  to  an  elementary  classroom 
environment.  The  sharing  of  resources  really  helped  me  see  others’  amazing  computer  talents  as  well  as 
organization  skills. 

The  assignments  in  this  course  gave  me  the  opportunity  to  create  projects  using  the  Web.  Interacting  with 
classmates  through  discussions  within  modules  was  enlightening  and  reaffirming. 

In  conclusion,  the  findings  of  this  study  indicate  that  in  order  to  establish  social  presence  and  active 
learning  community  for  ALN  courses,  the  systematic  approach  with  a variety  of  strategies  incorporating  with 
teaching  presence  should  be  designed  and  implemented.  The  effectiveness  of  STEP  approach,  which  includes:  the 
scaffolding  before  starting  new  learning  topics;  the  transaction  during  the  learning  process;  the  evaluation  during 
and  after  each  learning  topic;  and  the  presentation  of  outcomes  online,  has  been  shown  in  this  study.  Specifically, 
the  results  of  this  study  reveal  that  students  perceive  extremely  positive  on:  emotional  expression,  open 
communication,  and  group  cohesion  in  an  asynchronous  text-based  course. 

This  study  yields  results  consistent  with  previous  research  related  to  social  presence  and  teaching  presence 
in  asynchronous  learning  environment  (Garrison,  Anderson,  and  Archer,  2000;  Anderson,  Rourke,  Garrison,  and 
Archer,  2001;  Rourke,  Anderson,  Garrison,  and  Archer,  2001).  Instructors  of  asynchronous  learning  courses  can 
design  and  manage  learning  sequences,  provide  subject  matter  expertise,  establish  a positive  learning  environment, 
and  facilitate  active  learning. 
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Abstract 

Recent  widespread  use  of  hypermedia  in  education  may  be  challenging  to  some  learners  because  of 
information  overload,  technical  novelty,  and  multiple  levels  of  decision  making,  requiring  their  more  self-regulatory 
learning  ability.  In  scaffolding  learners  to  be  more  self-regulated,  it  would  be  critical  to  expand  our  understanding 
about  how  learning  with  hypermedia  relates  to  self-regulation.  This  paper  reviews  major  theories  of  self-regulation 
and  discusses  recent  research  findings  and  their  implications  for  facilitating  undergraduates’  self-regulated  learning 
with  hypermedia. 


Introduction 

Learning  with  hypermedia  has  the  potential  to  help  learners’  understanding  of  complex  knowledge  because 
it  enables  learners’  flexible,  dynamic,  and  non-linear  access  to  a wide  range  of  information  represented  as  text, 
graphics,  animation,  audio,  and  video  (Jacobson  & Archodidou,  2000;  Jonassen,  1996).  In  addition,  hypermedia 
allow  learners’  control  over  access,  and  this  increased  control  may  motivate  learning  (Dillon  & Gabbard,  1998; 
Kinzie,  1990;  Steinberg,  1989). 

Despite  these  benefits  of  learning  with  hypermedia,  interacting  with  hypermedia  may  be  challenging  to 
some  learners,  particularly  novice  learners  in  an  academic  domain,  because  of  information  overload,  technical 
novelty,  and  required  multiple  levels  of  decisions  making.  These  difficulties  of  hypermedia  integrated  learning 
probably  require  a more  strategic  learning  ability  from  learners.  According  to  Zimmerman  (1989),  self-regulated 
learning  occurs  to  the  degree  that  a student  can  use  personal  process,  such  as  goal  settings,  self-efficacy  perception, 
and  metacognitive  process,  to  strategically  regulate  behaviors  in  the  immediate  learning  environment.  Therefore 
self-regulated  learners  are  goal-oriented,  and  tend  to  monitor  their  academic  performance  to  evaluate  their  own 
progress  against  the  learning  goals  or  objectives. 

In  a hypermedia  learning  context,  learners  have  to  frequently  decide  which  content  to  branch  and  what 
scope  to  limit  the  content.  Therefore,  it  is  important  to  help  learners  promote  their  cognitive  and  metacogntive 
control  over  their  learning  that  is  characterized  by  the  behaviors  of  planning,  monitoring,  and  evaluating  their  own 
learning  process.  However  there  seems  to  be  lack  of  understanding  about  how  learners  can  learn  effectively  in  such 
environments  (Azevedo  & Cromley,  2003). 

In  respond  to  this  need  for  expanding  our  understanding  about  effective  learning  and  instruction  with 
hypermedia,  this  paper  discussed  on  theoretical  assumptions  of  self-regulation  and  the  function  of  self-regulation  in 
various  hypermedia  learning  contexts.  Additionally  recent  research  findings  were  overviewed  and  their  implications 
for  facilitating  undergraduates’  self-regulated  learning  were  discussed. 

Theoretical  Foundations 


Self-regulation 

Learners  are  self-regulated  to  the  degree  that  they  are  metacognitively,  motivationally,  and  behaviorally 
active  participants  in  their  own  learning  process  (Zimmerman,  1986).  According  to  Zimmerman  (2001)  self- 
regulation involves  awareness  of  the  potential  usefulness  of  self-regulation  processes  in  enhancing  their  academic 
achievement,  monitoring  of  the  effectiveness  of  learning  methods  and  strategies,  and  responding  to  this  feedback 
either  perceptually  or  behaviorally.  Self-regulation  also  entails  self-evaluation  of  capabilities  and  progress  in 
information  acquisition,  which  is  judged  against  their  own  goals  or  standards  (Schunk,  2004). 

Concerning  how  and  why  students  choose  to  use  particular  self-regulated  process,  strategy,  or  response, 
there  are  different  theoretical  perspectives.  From  the  behaviorally  oriented  view,  all  self-regulated  learning 
responses  are  under  the  external  control  of  reward  or  punishment  contingencies;  from  the  cognitive  perspective, 
students  are  motivated  primarily  by  a sense  of  self-esteem  or  self-concept;  from  the  social  cognitive  perspective, 
effective  self-  regulation  demands  goals  and  motivation. 


504 


Information  Processing  Perspectives 

From  the  information  processing  perspective,  learning  is  a set  of  processes  by  which  we  acquire 
information  (Winne,  2001).  Information  is  processed  in  three  locations:  sensory  register,  working  memory,  and 
long-term  memory.  Once  the  new  information  received  through  the  sensory  register  is  attended  to  and  perceived, 
working  memory  processes  the  input  by  rehearsing  or  associate  it  with  information  in  long-term  memory.  Since 
working  memory  is  very  limited  in  capability  and  duration,  information  is  organized  and  encoded  in  a way  to 
accommodate  retrieval  in  long-term  memory.  Long-term  memory  forms  network  of  nodes  and  links  of  chunks  of 
knowledge. 

We  can  retrieve  information  back  to  working  memory  through  spreading  activation  (Anderson,  1991).  A 
knowledge  unit  in  long  term  memory  is  connected  to  other  knowledge  units  relevant  to  it  forming  a knowledge 
network.  When  a knowledge  unit  is  activated,  it  spreads  to  all  the  other  units  connected  to  it.  Metacognitive 
monitoring  and  control  involve  all  steps  of  the  memory  process;  attention,  perception,  encoding,  and  recall.  Over  the 
memory  process,  metacognition  plays  a critical  role  in  regulating  acquisition  of  information.  Metacognition  has  been 
simply  defined  as  thinking  about  thinking  (Hacker,  1998).  According  to  Flavell  (1976),  metacognition  is  “knowledge 
and  cognition  about  cognitive  phenomenon”  (p.906)  and  involves  deliberate,  intentional,  and  goal-directed  mental 
operations.  Metacognition  occurs  through  actions  and  interactions  among  four  components:  metacognitive 
knowledge,  metacognitive  experiences,  goals,  and  strategies. 

Hacker  (1998)  described  four  components  of  Flavell’s  Metacognition:  1)  Metacognitive  knowledge 
including  a learner’s  stored  world  knowledge,  their  cognitive  tasks,  goals,  strategies  for  achieving  them,  actions,  and 
experiences;  2)  metacognitive  experiences  including  one’s  awareness  of  his  cognitive  and  affective  process  and 
how  the  progress  toward  the  goal  of  a current  process  is  being  made;  3)  Adaptation  of  metacognitive  knowledge  in 
response  to  the  metacognitive  experiences;  4)  Adjustment  of  the  goal  or  the  strategies  to  be  used  to  accomplish  the 
goal.  From  Flavell’s  view,  self-regulation  is  metacognitive  control  over  the  learning  process. 

On  the  basis  of  the  information  processing  perspective,  the  four  phases  Self-Regulated  Learning  (SRL) 
model  was  recently  introduced  (Winnie  and  Hadwin,  1998;  Winne,  2001).  The  SRL  model  includes  four 
components:  conditions,  products,  standards,  and  evaluations.  Conditions  refer  to  resources  available  for  work  on  a 
task  and  constraints  that  may  affect  information  process.  Among  the  resources,  the  most  significant  resource  is 
considered  prior  knowledge.  The  second  component,  products,  is  new  information  created  when  information  is 
processed.  Successive  products  build  toward  the  goal,  which  may  reside  in  memory  or  in  the  environment  as  overt 
outcomes.  The  third  component  is  standards  (goals)  that  current  learning  processes  are  judged  against.  The  final 
component  is  evaluations,  which  are  made  by  the  monitoring  process. 

All  components  go  through  four  phases:  1)  Defining  the  task,  2)  setting  goals  and  planning,  3)  enacting 
tactics,  and  4)  adapting  metacognition.  In  the  defining  the  task  phase,  the  learner  processes  information  about  the 
conditions  that  characterize  the  learning  task.  In  the  setting  goal  and  planning  phase,  the  learner  frames  a goal  and 
specifies  plans  to  approach  it.  In  the  enacting  tactics  phase,  the  learner  applies  tactics  and  strategies  that  were 
identified  in  the  setting  goal  and  planning  phase.  In  the  adapting  metacognition  phase,  the  learner  makes  major 
adaptations  to  their  schema  that  structure  how  self-regulating  is  carried  out. 

Constru  ctivist  Per  speed  ves 

Vygosky’s  zone  of  proximal  development  (ZDP)  strongly  influences  self-regulation.  ZDP  is  defined  as  “the 
distance  between  the  actual  developmental  level  as  determined  by  independent  problem  solving  and  the  level  of 
potential  development  as  determined  through  problem  solving  under  guidance  or  in  collaboration  with  more  capable 
peers”(Vygotsky,1978,  p.86,cited  by  Schunk,  2004).  In  the  ZDP,  an  adult  with  expertise  and  a learner  work  together 
on  a task  that  the  learner  could  not  perform  independently  because  of  the  difficulty  level  (Schunk,  2004). 

Instructional  scaffolding  refers  to  the  process  of  controlling  task  elements  that  are  beyond  the  learners’ 
capabilities  so  that  they  can  focus  on  and  master  those  features  of  the  task  that  they  can  grasp  quickly  (Burning, 
Schraw,  & Ronning,  1999).  In  this  approach,  a teacher  provides  support,  and  gradually  fades  aid  as  the  learners 
acquire  the  skill  and  control  of  their  own  learning. 

Implicit  theories  bring  a new  perspective  to  self-regulation.  Learners  construct  theories  about  self-regulated 
learning  (Schunk,  2004);  therefore  self-regulation  is  highly  contextualized  (Paris,  Byrnes,  & Paris  2001).  Learners 
can  formulate  theories  about  what  contributes  to  success  in  performing  different  tasks  (Schunk,  2004).  Self- 
regulatory  strategies  that  the  learners  bring  to  their  learning  might  be  either  effective  or  misleading.  Learners  might 
believe  they  are  capable  of  learning  some  tasks  or  they  might  doubt  their  ability  to  complete  them.  The  ideas 
underlying  implicit  theories  suggest  that  it  is  important  for  educators  to  understand  what  beliefs  and  feelings  that  the 
learners  initially  have  about  their  own  learning  of  specific  tasks  so  that  any  instructional  intervention  for  self- 
regulated  learning  is  adjusted  to  the  needs  of  individual  learners. 
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Goal  Orientation  and  Self-regulation 

In  addition  to  learner’s  beliefs  and  needs,  learners’  goal  orientation  can  also  affect  what  and  how  learners 
approach  learning  material.  The  type  of  goal  that  a student  adopts  may  impact  the  student’s  perception  of  progress, 
self-efficacy,  and  self-evaluation  (Schunk,  2004).  Researchers  have  identified  two  different  goal  orientations,  task- 
mastery  and  performance  goals.  Task-mastery  goals  focus  students’  attention  on  processes  and  strategies  that  help 
them  acquire  knowledge,  behaviors,  skills,  or  strategies.  On  the  other  hand,  performance  goals  direct  students’ 
attention  to  completing  tasks.  Research  supports  that  pursuing  task-mastery  goals  enhance  intrinsic  motivation  and 
active  cognitive  involvement  characterized  by  self-regulatory  activities  (Meece,  Blumenfeld,  & Hoyle,  1988) 

Hypermedia  and  Self-regulation 

Hypermedia  learning  environments  enable  either  linear  or  non-linear  presentation  of  knowledge. 
Knowledge  presented  in  a non-linear  fashion  allows  learners  to  navigate  freely  from  one  concept  to  another,  without 
concern  for  predetermined  order  or  sequence  (McManus,  2001).  Linked  structure  of  hypermedia  may  accurately 
represent  the  associative  network  that  makes  up  the  human  memory  and  may  allow  realistic  representation  of  expert 
semantic  network  (Jonassen,  1993).  Semantic  networks  and  concept  maps  represent  the  structure  of  the  information 
within  the  hypermedia  environments  (Reed  & Steven,  1995).  However,  the  knowledge  representations  that  learners 
construct  from  hypermedia  may  be  incomplete  resulting  in  misconceptions  or  misunderstanding,  unless  the  learners 
are  provided  with  additional  modeling  or  instruction  about  the  knowledge  structure  is  provided. 

Additionally  the  non-linear  trait  of  hypermedia  learning  environments  lets  the  learners  control  the 
organization  of  the  information  (Liu,  1992). Thus  the  learning  situations  with  more  control  and  options  involve 
multiple  levels  of  decision  making  , probably  requiring  higher  level  of  ability  to  control  learning  processes  from  the 
learners. 

As  referred  in  the  previous  section,  regulating  learning  involves  learners’  metacognitive  awareness  of  their 
learning  process.  Metacognitive  awareness  is  possible  when  the  learners  have  conditional  knowledge,  knowledge 
about  when  and  how  to  use  declarative  or  procedural  knowledge  relevant  to  a given  learning  task.  Conditional 
knowledge  allows  learners  to  metacognitively  monitor  their  learning  progress  toward  their  learning  goals  and 
evaluate  the  effectiveness  of  their  learning  process  against  the  goals.  The  self-regulatory  learning  process  in 
hypermedia  environments  seems  to  be  a dynamic  process  involving  interaction  among  learners’  self-regulatory 
ability,  learners’  prior  knowledge,  traits  of  hypermedia,  and  the  learning  tasks.  While  monitoring  their  learning 
processes  with  hypermedia,  learners  change  or  revise  their  learning  strategies  or  tactics  to  make  their  learning  more 
effective.  This  also  implies  that  research  can  be  designed  taking  into  account  interaction  of  such  diverse  factors. 

In  the  following  review,  special  attention  was  paid  to  determining  how  the  studies  handled  those  factors 
relevant  to  self-regulation  in  hypermedia  learning  environments  and  what  the  findings  about  the  relationship  among 
those  factors  are,  and  what  their  findings  imply  for  helping  self-regulatory  learning.  The  studies  included  in  this 
review  were  selected  because  they  all  related  to  undergraduates’  self-regulatory  learning  with  hypermedia. 

However,  the  included  studies  do  not  exhaust  all  studies  pertaining  to  undergraduates’  self-regulatory 
learning  in  hypermedia  learning  settings. 

Current  Research  on  Self-regulation  and  Hypermedia 

Hypermedia  environments  can  provide  learners  with  more  freedom  to  control  pace,  order,  and  sequence  of 
learning  (Liu,  1992),  with  more  autonomy,  some  learners  in  hypermedia  environments  may  feel  disorientated  or  less 
efficacious.  Learning  with  hypermedia  would  be  challenging  especially  to  the  learners  who  have  less  domain 
knowledge.  In  deriving  implications  for  effective  instruction  for  learning  with  hypermedia,  current  research  appears 
to  focus  on  addressing  two  dominant  aspects:  1)  relationship  between  the  feature  of  hypermedia  and  self-regulation 
and  2)  the  effects  of  various  scaffolds  on  promoting  self-regulation  ability  and  learning.  Thus,  this  paper  reviewed 
several  studies  that  were  thought  to  dominantly  address  those  two  aspects.  The  included  studies  were  issued  from 
2001  to  2005  and  all  relate  to  learning  with  hypermedia  of  undergraduates  who  were  novices  in  a content  domain. 


Non-linear  Structure  of  Hypermedia  and  Self-regulation 

When  the  demands  that  a task  creates  on  a learner  are  high,  or  the  task  is  complicated,  the  learner’s 
working  memory  is  overloaded,  leaving  little  and  sometimes  insufficient  resources  to  apply  to  self-regulating 
(Zimmerman,  2001).  Particularly  in  hypermedia  learning  environment,  the  cognitive  load  that  the  non-linear 
structure  of  hypermedia  may  impose  on  learners  would  be  a great  concern. 
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Eveland  and  Dunwoody  (2001)  tested  if  hypermedia  learning  environments  bring  higher  cognitive  load 
than  traditional  print-based  learning  environments.  In  this  study,  levels  of  cognitive  load  were  conceptualized  as 
being  directly  consistent  with  the  variation  of  non-linearity  in  the  Web.  Different  levels  of  media  conditions,  control, 
print,  linear  web,  non-linear  web  and  advisement  web,  were  compared  on  the  performances  of  recognition  and  cued 
recall  respectively  as  measuring  of  learning.  Motivation,  web  expertise,  and  self-reported  cognitive  load  were  taken 
into  account  as  covariates.  The  result  revealed  that  learning  was  greater  for  print  than  the  web  conditions  on 
recognition  performance  indicating  probably  lower  cognitive  load  in  the  print  condition  than  other  media  conditions. 
To  the  contrary,  there  were  no  significant  differences  across  media  conditions  on  the  recall  performance.  As 
expected,  self-reported  cognitive  load  negatively  predicted  the  recall  performance.  Based  on  the  findings,  the  author 
suggested  that  there  was  evidence  to  support  the  claim  that  hypermedia  presentations  may  reduce  learning  compared 
to  traditional  print  because  of  such  factors  as  cognitive  load  (Oliver,  1996)  and  disorientation  (Calvi,  1997;  Darken 
& Sibert,  1996;  MacDonald  Stevenson,  1996).  The  result  from  the  self-report  about  cognitive  load  also  confirmed 
that  non-linear  feature  of  hypermedia  could  be  an  obstacle  to  self-regulatory  learning  requiring  learners’  extra 
cognitive  efforts  of  navigating  from  a part  of  content  to  another.  However,  this  finding  is  not  conclusive  because 
there  might  be  other  important  variables  that  could  influence  learning.  For  example,  learners  possibly  perform  better 
because  of  their  prior  knowledge  about  the  content  (reading  text)  rather  than  different  media  conditions. 

In  this  study  the  prior  knowledge  about  the  content  of  interest  was  not  directly  measured  across  the 
conditions.  In  addition,  the  failure  in  finding  any  difference  among  various  media  conditions  on  recall  measuring 
might  be  because  variation  of  the  level  of  linearity  across  different  media  conditions  were  not  be  optimally 
established  to  determine  any  dissimilarity  among  them.  Another  possibility  is  that  the  reading  material  might  not 
actually  be  differed  in  the  level  of  freedom  of  control  over  navigation  across  various  media  conditions.  As  the  author 
indicated,  the  perceived  overload  and  actual  cognitive  overload  are  possibly  unlike. 

While  Eveland  and  Dunwoody(2001)’s  study  focused  on  the  feature  of  hypermedia  and  learning, 
McManus(2000)  examined  a more  direct  relationship  between  the  non-linear  structure  of  hypermedia  and  self- 
regulation. This  study  examined  how  non-linearity  and  an  advance  organizer  together  relate  to  self-regulation  and 
learning.  The  results  illuminated  two  types  of  interaction;  one  was  the  interaction  between  non-linearity  and  self- 
regulation (p — 0.054).  and  the  other  was  the  interaction  between  non-linearity  and  advance  organizer  (p — 0.052).  It 
was  suggested  that  highly  self-regulated  learners  learn  poorly  in  mostly  linear  hypermedia  learning  environments 
where  they  have  very  few  choices,  while  medium  self-regulated  learners  learn  less  in  highly  non-linear 
environments  where  there  are  too  many  options.  The  results  also  suggested  that  an  advance  organizer  is  more 
effective  in  highly  non-linear  web-based  hypermedia  than  in  mostly  linear  learning  environments. 

The  results  from  the  McManus  (2000)’s  study  implied  importance  of  adapting  hypermedia  to  learners’  self- 
regulation ability.  A diagnosis  of  learners’  self  regulation  ability  should  precede  designing  a learning  unit  with 
hypermedia.  Additionally,  repeated  diagnoses  of  the  learners’  progress  in  their  ability  to  regulate  learning  as  well  as 
assessment  made  prior  to  implementing  a learning  unit  would  be  necessary  particularly  when  learners’  consistent 
change  is  anticipated.  For  example,  if  a part  of  the  learning  goal  is  learning  about  hypermedia  technology  itself,  the 
learner  will  need  less  scaffolding  or  guidance  as  their  expertise  grows  with  the  on-going  use  of  the  hypermedia.  The 
non-linearity  of  hypermedia  would  become  less  of  a cognitive  load  with  time. 

The  variation  of  self-regulation  may  be  associated  with  the  ways  the  learners  navigate  hypermedia  in  a 
particular  manner.  Sweany  (2000)  attempted  to  determine  if  the  particular  pattern  of  hypermedia  navigation  interacts 
with  strategic  learning.  This  study  investigated  what  the  emerging  patterns  in  undergraduate  students’  navigation 
behaviors  are  and  how  they  are  associated  with  strategic  learning  while  the  learners  interact  with  material  including 
a CD-Rom  and  the  Internet  designed  for  an  introductory  French  course.  For  the  navigation  cluster  analysis,  the  log 
files  were  automatically  recorded  while  the  students  were  using  the  computer  in  the  class,  and  analyzed.  Goal 
orientation,  instrumentality,  and  metacognitive  self-regulation  were  regrouped  as  a strategic  learning  variable.  It 
was  anticipated  that  combination  of  strategic  learning  and  a particular  pattern  in  using  computer  features  would  have 
the  highest  relationship  with  achievement. 

This  study  revealed  that  students'  achievement  differed  with  the  ways  they  navigate  hypermedia.  Based  on 
these  findings,  it  was  suggested  that  strategic  learning  might  mediate  the  relationship  between  hypermedia 
navigation  and  achievement.  As  the  author  noted,  not  all  students  will  automatically  use  hypermedia  programs 
effectively.  Therefore,  it  would  be  important  to  help  learners  to  be  more  strategic  in  using  hypermedia  integrated 
learning  contexts.  Furthermore,  navigational  design  should  be  considered  prior  to  integrating  hypermedia  into  the 
classroom. 

Puntambekar  and  Stylianou  (2005)  similarly  attempted  to  examine  the  ways  that  students  navigated  through 
hypermedia.  Although  the  study  did  not  target  undergraduate  subjects,  the  study  seems  to  provide  insights  into 
learning  with  hypermedia  across  diverse  levels  of  subjects.  The  study  found  the  positive  effect  of  metacognitive 
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navigational  scaffolding  on  students’  understanding  of  organization  of  information  and  suggested  that  the 
metacognitive  prompts  reminded  students  of  the  goal  and  encouraged  them  to  reflect  on  their  decisions  they  made 
for  seeking  information.  In  a hypermedia  system  a student  constantly  makes  decisions  about  which  link  to  visit  next 
and  about  what  piece  of  information  is  relevant  to  their  current  goals  of  learning.  Thus,  it  would  important  for 
readers  to  monitor  and  regulate  both  their  comprehension  and  navigation  strategies  (Puntambekar  & Stylianou, 
2005). 

There  is  also  empirical  evidence  that  learners  perceive  metacognitive  structural  cues  distracting  rather  than 
helpful  (Lee  et  al.,  1997).  Processing  metacognitive  scaffolds  may  take  additional  cognitive  efforts  from  learners, 
therefore  adding  more  cognitive  load  rather  than  promoting  learning.  As  a result,  in  designing  navigational  support 
tools,  it  should  be  considered  how  to  minimize  the  demands  from  such  structural  cues. 

An  issue  raised  from  this  study  was  the  potential  mediation  of  the  students’  prior  knowledge  on  the  content 
topic  of  interest.  The  navigational  behaviors  may  be  associated  with  the  student’s  semantic  organization  of  their 
existing  domain  knowledge.  If  this  is  the  case,  how  should  hypermedia  semantically  intervene  individual  domain 
knowledge  in  such  a way  that  further  learning  occurs  more  effectively  with  hypertext?  A particular  issue  would  be 
how  to  tailor  self-regulatory  monitoring  support  to  different  students  and  when  is  the  right  time  for  providing  it  (e.g., 
providing  whenever  asked  by  the  learners  or  automatically  after  certain  amount  of  time  has  past)? 

Effective  Scaffolding  of  Self-regulation 

The  studies  previously  reviewed  indicated  that  learners  might  be  cognitively  overloaded  or  disoriented  with 
the  non-linear  structure  of  hypermedia.  Additionally,  it  was  suggested  that  goal-based  navigation  and 
matanavigational  scaffolding  intended  to  promote  metacognitive  monitoring  of  navigational  behaviors  enhanced 
learning  in  the  hypermedia  learning  environments.  In  the  following  section,  how  self-regulatory  scaffolding  learning 
with  hypermedia  influences  self-regulation  and  learning  were  reviewed  based  on  recent  findings. 

Debowski,  Wood,  and  Bandura  (2001)  examined  the  impact  of  guidance  over  search  of  electronic  database 
on  self-regulation.  In  this  study,  self-efficacy,  satisfaction,  and  internal  motivation  were  conceptualized  as  major 
predictive  factors  for  self-regulation  on  the  student’s  performance.  It  was  found  that  the  guided  search  mode  of 
electronic  database  was  more  effective  than  the  self-guided  search  mode  in  developing  post-training  self-efficacy 
and  satisfaction;  however,  intrinsic  motivation  was  not  affected  by  exploration  modes.  The  guided  search  mode 
included  instruction  on  the  order  for  completing  database  searching  tasks,  reminders  of  the  steps  to  follow  in 
developing  a strategy  for  each  task,  and  modeling  of  responses  that  followed  problems  and  mistakes  in  the  use  of 
strategies  during  practice.  On  the  other  hand,  in  the  self-guided  (or  enactive)  search  mode  the  sequence  of  tasks,  the 
steps  in  the  strategy  process,  and  the  responses  to  problems  were  left  to  the  discretion  of  the  individual  searcher.  It 
was  revealed  that  guided  search  produced  higher  levels  of  perceived  self-efficacy,  satisfaction,  strategy  quality,  and 
performance  and  a lower  level  of  wasted  effort  on  electronic  search  tasks  than  self-guided  search  did.  The  author 
concluded  that  self-enactive  learning  would  be  most  effective  when  structured  tasks  are  given  to  learners  who  have 
greater  expertise  on  dynamic  tasks  along  with  more  informative  feedback.  Among  theorists,  goal  setting  and 
planning  were  considered  as  critical  stages  for  self-regulated  learning.  Social  cognitive  theorists  claim  that  self-set 
goals  promote  self-efficacy.  Azevedo  et  al.  (2002)  examined  how  the  different  instructional  goal  setting  conditions 
influence  students’  learning  of  complex  science  concepts  with  hypermedia  based  on  the  SRL  model  (Winnie  et  al., 
1998;  2001).  The  SRL  model  explains  learners’  recursive  cycles  of  control  and  monitoring  during  the  four  phases  of 
learning:  perceiving  tasks,  setting  goals  and  planning,  employing  tactics,  and  enacting  tactics. 

In  this  study  forty  undergraduates  were  randomly  assigned  to  four  different  goal  setting  conditions:  the 
bottom-up  condition,  the  learner-generated  sub  goal  condition,  the  strategy  instruction  condition,  and  the  co- 
regulation condition.  The  pre-  and  post  -tests  comparison  group  design  with  a think-aloud  methodology  was  used.  It 
was  revealed  that  both  the  co-regulation  (with  tutor’s  guidance)  and  strategy  instruction  conditions  facilitated 
students’  learning.  The  data  from  all  conditions  were  analyzed  over  the  five  main  SRL  categories:  planning, 
monitoring,  strategy  use,  handling  task  difficulty  and  demands,  and  interest.  Findings  revealed  that  the  co-regulated 
goal  setting  condition  promoted  learning  of  complex  science  contents.  In  the  co-regulation  condition,  the  tutor 
assisted  learners  in  regulating  their  learning  by  using  various  strategies,  scaffolding  their  learning,  and  intervening 
when  they  asked  for  assistance.  The  tutors  helped  the  students  plan  sub-goals,  monitor  their  cognitive  processes,  and 
provided  feedback 

In  the  subsequent  study,  Azevedo  and  colleagues  (2003)  examined  the  effectiveness  of  adaptive 
scaffolding  on  self-regulated  learning.  The  subjects  were  assigned  to  one  of  three  conditions:  adaptive  content  and 
process  scaffolding  (ACPS),  adaptive  process  scaffolding  (APS),  and  no  scaffolding  (NS)  and  were  trained  to  use  a 
hypermedia  environment  to  learn  about  the  circulatory  system.  In  the  adaptive  scaffolding  condition,  students  could 
access  a tutor  who  provided  scaffolding  to  help  with  their  content  understanding  (content  scaffolding)  or  self- 
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regulated  learning  process  (processing  scaffolding).  Based  on  the  analyses  from  the  pre-and  post-tests,  and  verbal 
protocol  data,  the  author  reported  that  the  adaptive  scaffolding,  both  ACPS  and  APS,  facilitated  learning  more  than 
NS.  An  interesting  finding  from  the  data  was  the  differences  of  self-regulatory  behaviors  while  learning  between 
the  two  adaptive  conditions.  Participants  in  the  ACPS  condition  regulated  their  learning  by  engaging  in  help- 
seeking and  over-relying  on  the  tutor  to  regulate  their  learning.  While,  participants  in  the  APS  condition  regulated 
their  learning  by  planning  and  monitoring  their  emerging  understanding,  and  handled  task  difficulties  well. 

In  summarizing  the  studies  conducted  by  Azevedo  and  his  colleagues,  it  was  suggested  that  scaffolding 
learners’  use  of  self-regulation  strategies  in  hypermedia  learning  environments  is  critical.  In  addition,  such 
scaffolding  would  be  most  effective  if  adapting  to  learners’  individual  needs  in  controlling  their  learnt  process. 
However,  the  scaffolding  should  not  be  in  excess  because  support  exceedingly  provided  may  ruin  the  learners’ 
desire  to  be  active  in  controlling  their  learning  process.  How  learners’  prior  content  knowledge  is  associated  with 
use  of  self-regulation  strategies  still  remains  to  be  examined  in  future  studies. 

The  use  of  the  SRL  strategies  has  been  suggested  to  be  context  dependent.  Whipp  and  Chiarelli  (2005) 
found  that  students  adapted  their  use  of  self-regulation  strategies  to  the  unique  feature  of  the  web-based  learning 
environment.  Paralleling  this  finding,  the  study  conduced  by  Dabbagh  & Kitsantas  (2005)  discovered  that  learners 
facilitated  or  withdrew  their  intention  to  use  SRL  strategies  in  accordance  to  the  unique  features  of  learning 
environments.  In  this  study  it  was  determined  that  in  distributed  (blended)  learning,  students  differed  their  use  of 
self-regulation  process  depending  on  the  types  of  web-based  pedagogical  tools  provided. 

Self-regulation  is  suggested  to  be  particularly  important  in  distributed  learning  settings  because  the  students 
are  often  asked  to  complete  tasks  with  limited  interaction  and  support  from  the  instructor  and  peers  (Kauffman, 

2002;  Whipp  & Chiarelli,  2005).  Distributed,  or  blended,  learning  situations  as  rich  hypermedia  learning  contexts, 
involves  such  features  as  enabling  access  of  information,  navigation  through  content  links,  web-links  and 
information  spaces,  and  the  use  of  search  engines  (Dabbagh  & Kitsantas,  2005). 

Dabbagh  and  Kitsantas’  study  investigated  how  different  categories  of  Web-Based  Pedagogical  Tools 
(WBPT)  integrated  into  a Web-CT  course  management  system  (CMS)  supported  different  SRL  processes. 
Additionally,  they  attempted  to  determine  students’  perception  of  the  usefulness  of  WBPT  in  evoking  the  SRL 
processes.  The  findings  indicated  that  content  creation  and  delivery  tools  supported  the  SRL  processes  of  goal 
setting,  help  seeking,  self-evaluation,  and  task  strategies.  While  collaborative  and  communication  tools  (e.g.,  email 
and  discussion  forum),  content  creation  delivery  tools  (e.g.,  tools  for  uploading  contents  and  assignments  and  tools 
for  students  to  access  course  resources  and  readings)  supported  goal  setting,  time  planning  and  management,  and 
help  seeking.  Administrative  tools  (e.g.,  tools  to  manage  general  course  information  and  functions;  and  student 
information,  and  interactions)  supported  self-monitoring,  self-evaluation,  time  planning  and  management,  and  help 
seeking  while  assessment  tools  (tools  for  announcing  or  tracing  student  progress)  supported  task  strategies,  self 
monitoring,  and  self-evaluation. 

Based  on  the  qualitative  analysis  it  was  suggested  that  when  engaging  students  in  problem  solving  tasks  in 
distributed  learning  environments,  content  delivery  tools  should  be  used  to  provide  a variety  examples  or  sample 
solutions  to  enhance  students’  self-regulatory  learning.  The  qualitative  results  also  revealed  that  the  students 
perceived  that  content  creation  and  delivery  tools,  especially  the  reading,  resources,  and  assignment/rubrics  features, 
were  useful  in  scaffolding  help  seeking  and  self-evaluation  while  completing  assignments  involving  dialogical 
learning  tasks.  The  author  suggested  that  when  engaging  students  in  articulation  and  refection  required  in  on-line 
discussions,  content  and  delivery  tools  should  provide  relevant  resources  and  allow  an  on-line  access  to  the  readings. 
Additionally  providing  students  a rubric  of  how  they  will  be  evaluated  was  considered  to  be  important  in  scaffolding 
the  SRL  process  of  self-evaluation. 

Other  implications  derived  from  the  findings  included  that  a group  discussion  area  should  be  provided  to 
facilitate  help  seeking,  time  planning,  and  time  management.  It  also  found  that  a space  to  post  working  drafts 
should  be  provided  to  support  evaluation  and  task  strategies.  A rubric  should  be  available  to  help  students’  goal 
setting,  and  an  access  to  resources  should  be  made  available  to  support  the  help  seeking  process.  Students  perceived 
that  administrative,  assessment,  and  collaborative  and  communication  tools  (e.g.,  e-mail  and  discussion  features 
tools)  were  useful  in  time  planning  and  time  management,  and  self-monitoring. 

However,  the  findings  from  Dabbagh  and  Kitsantas’  study  may  be  applied  restrictedly  to  a web-based 
course  that  includes  face  to  face  interactions  and  also  uses  a CMS  tool  similar  to  Web-CT.  Also,  learners’  self- 
report  about  use  of  WBPT  is  probably  dissimilar  to  their  actual  use  of  WBPT.  The  study  did  not  follow  students’ 
actual  use  of  WBPT  in  terms  of  frequency;  therefore,  it  was  impossible  to  validate  students’  self-report  with 
behavioral  data. 

Choi,  Land  and  Turgeon  (2005)  tested  the  effect  of  a scaffolding  tool  on  self-regulatory  learning  in  a totally 
on-line  learning  context.  This  study  attempted  to  determine  if  scaffolded  peer  questioning  in  on-line  discussion 
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facilitated  metacognition  and  learning  Meaningful  verbal  interactions  and  generating  adapted  feedback  require  a 
certain  level  of  domain  and  metacognitive  knowledge  from  a questioner.  A learner  in  a new  domain  is  often  limited 
in  domain  and  metacognitive  knowledge;  therefore  they  can  neither  ask  the  right  questions  nor  generate  constructive 
feedback  (Land,  2000). 

This  study  used  a time-series  control-group  design.  The  study  included  and  conducted  in  online 
introductory  class  targeted  learning  for  turf-grass  management  with  thirty  nine  undergraduates  as  the  subjects.  The 
results  showed  that  there  was  a significant  effect  of  use  on  frequency  of  peer-generated  questions.  However  the 
peer-questioning  scaffolds  did  not  improved  the  quality  of  questions.  The  effect  seemed  to  be  dependent  on  the  type 
of  a question.  The  dominant  portion  of  the  peer  generated  questions  was  clarification  /elaboration  questions,  which 
was  defined  as  “questions  seeking  additional  information  to  clarify  or  elaborate  learners’  initial  ideas”.  Thus  the 
findings  were  limited  in  generalizing  other  types  of  questions,  such  as  counter-arguments  (peer-generated  ideas 
expressing  disagreement  with  learners’  initial  ideas,  which  could  address  cognitive  conflicts)  and  context-  or 
perspective-oriented  questions  (hypothetical  questions  changing  critical  factors  in  a given  problem  situation  or 
considering  different  perspectives  on  the  problems).  Further  research  may  concentrate  on  how  to  develop  more 
effective  peer-questioning  scaffolds  to  address  diverse  types  of  questions. 

In  general,  findings  from  Choi  and  colleagues’  study  indicated  that  fixed  (as  apposed  to  dynamic)  scaffolds 
were  useful  to  increase  the  frequency  of  student  questioning  behavior  during  online  discussion.  Further  studies  may 
want  to  focus  on  the  quality  improvement  of  peer-generated  questions  using  dynamic  and  adaptive  scaffolding.  As 
the  author  addressed  how  those  factors  (prior  knowledge,  metacognition,  task  complexity,  and  scaffolding  type) 
interact  in  an  online  learning  context  would  be  an  important  research  task  in  the  future. 

Conclusion 

The  studies  reviewed  in  this  paper  shared  the  idea  that  despite  of  the  usefulness  of  hypermedia  because  of 
its  ability  to  display  knowledge  in  a non-linear  way,  hypermedia  integrated  learning  may  be  challenging  to  novice 
learners  in  a content  domain.  Therefore,  learning  with  hypermedia  may  require  higher  self-regulation  ability  from 
learners.  The  findings  from  the  studies  included  indicated  that  learners’  self  regulatory  ability  interacts  with  the 
feature  of  non-linearity  of  hypermedia.  In  addition,  the  ways  that  learners  use  their  self-regulation  ability  with 
hypermedia  is  highly  situational;  some  of  those  variables  that  probably  interact  with  personal  variables  in 
hypermedia  learning  were  support  from  teachers  and  peers,  teachers’  attitudes,  characteristics  of  content  or  tasks, 
and  instructional  strategies  intended  to  guide  informational  organization  (e.g.,  navigational  or  organization  clues).  It 
was  also  strongly  suggested  that  individualizing  scaffolding  for  the  students’  regulation  over  their  learning  is  critical 
to  promote  self-regulated  learning.  Although  results  from  the  studies  reviewed  need  to  be  further  tested,  a few 
important  implications  for  designing  instruction  in  hypermedia  learning  contexts  were  addressed. 

First  of  all,  the  findings  implied  that  there  were  positive  relationships  between  the  level  of  non-linearity  of 
hypermedia  and  cognitive  load  (Eveland  and  Dunwoody,  2001)  and  metacognitive  navigational  scaffolds  was  useful 
in  reducing  the  cognitive  load.  As  opposed  to  the  traditional  learning  settings  where  content  is  provided  in  a linear 
order  and  often  the  teacher  decides  what  sequence  the  learners  follow  to  complete  a task,  learners  in  hypermedia 
learning  contexts  themselves  should  deal  with  the  non-linearity  in  terms  of  the  organization  of  content  and  the 
sequence  for  completing  a task.  Thus,  learners  need  to  be  more  strategic  to  metacognitively  monitor  their  decisions 
for  paths  in  relation  to  the  learning  goal.  Particularly  the  studies  conducted  by  Puntambekar  and  Stylianou  (2005) 
and  McManus  (2000)  indicated  that  metacognitive  scaffolds  such  as  navigational  prompts  and  advanced  organizers 
were  effective  for  students’  understanding  of  organization  of  information  and  goal-oriented  navigational  behaviors. 

Another  important  implication  for  designing  instruction  integrating  hypermedia  was  that  when  instructional 
strategies  such  as  an  advance  organizer  or  navigational  clues  are  employed  as  scaffolding  tools,  the  level  of  non- 
linearity of  content  organization  and  learner’  ability  to  regulate  their  own  learning  should  be  taken  into  account  as 
the  major  concern.  According  to  McManus  (2002),  not  all  learners  seem  to  benefit  from  such  scaffolds  as  advanced 
organizers.  The  study  showed  that  highly  self-regulated  learners  learn  better  in  highly  non-linear  environments 
where  there  are  many  options  of  paths.  On  the  other  hand  less  self-regulated  learners  perform  better  in  relatively 
linear  hypermedia  learning  environments  with  few  options  of  paths.  This  suggested  that  designing  a course 
involving  hypermedia  should  be  adapted  to  individual  level  of  self-regulation.  The  study  also  suggested  that  an 
advanced  organizer  is  more  effective  in  a highly  non-linear  hypermedia  environment  than  in  a mostly  linear  learning 
environment.  As  mentioned  in  the  previous  section,  this  implies  constant  diagnoses  of  self-regulation  ability  of 
learners  should  precede  designing  a course  to  include  hypermedia.  Additionally  it  would  be  important  to  encourage 
students  to  monitor  their  understanding  and  use  of  self-regulatory  strategies  by  frequently  prompting  the  learning 
goals. 
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However,  the  potential  mediation  effect  of  students’  prior  knowledge,  such  as  goal  orientation, 
instrumentality,  and  metacognitive  self-regulation  as  Sweany  (2000)  determined,  on  students’  navigational 
behaviors  and  learning  still  remains  to  be  examined.  The  navigational  behaviors  may  be  associated  with  the 
student’s  semantic  organization  of  their  existing  domain  knowledge.  If  it  is  the  case,  how  to  semantically  intervene 
individual  domain  knowledge  in  such  a way  that  further  learning  occurs  more  effectively  with  hypermedia  should  be 
considered.  In  addition  there  is  empirical  evidence  that  learners  perceived  metacognitive  structural  cues  were 
distracting  rather  than  helpful  (Lee  et  al.,  1997).  Therefore,  when  including  a navigational  support  tool  necessary  for 
learners’  understating  of  information,  it  should  be  considered  how  to  minimize  cognitive  demands  from  such 
structural  cues  while  maintaining  the  benefits  from  those  tactics.  Particular  issues  addressed  were  how  to  tailor  self- 
regulatory  monitoring  support  to  different  learners  and  when  the  right  time  is  for  providing  it  (e.g.,  providing 
whenever  asked  by  the  learners  or  automatically  after  certain  amount  of  time  has  past). 

From  the  studies  conducted  by  Debowski  et  al  (2001)  and  Azevedo  et  al  (2002;  2003),  another  important 
implication  for  designing  instruction  with  hypermedia  was  addressed.  The  studies  implied  that  guidance  and 
informative  feedback  from  tutors  and  teachers  about  how  to  regulate  learning  was  effective  for  learners’  planning 
their  goals,  monitoring  their  emerging  understanding,  and  using  effective  strategies.  The  studies  also  indicted  that 
there  is  a positive  impact  of  guidance  on  learners’  self-regulation  in  hypermedia  environments.  Guided  search 
(Debowski,  Wood,  & Bandura,  2001)  produced  higher  levels  of  perceived  self-efficacy,  satisfaction,  strategy 
quality,  and  performance  over  self-guided  search.  A suggestion  was  that  self-enactive  learning  is  more  effective 
when  the  learners  are  more  trained  in  using  hypermedia,  the  tasks  are  structured  enough,  and  more  informative 
feedback  is  provided  to  the  learners.  The  informative  feedback  given  by  a teacher  or  tutor  is  critical  for  the  learner 
to  be  more  strategic  in  their  learning.  Simply  knowing  how  to  use  strategies  does  not  guarantee  that  students  will 
use  them,  therefore  it  is  important  to  help  learners  to  formulate  effective  strategies  for  their  learning  (Schunk,  2004) 
with  hypermedia.  Instructional  methods  and  educators’  attitudes  both  possibly  appeared  to  influence  learner’s  self- 
regulation in  hypermedia  environments. 

Research  also  suggested  that  self-regulation  is  context  dependent  (Whipp  & Chiarelli,  2005).  As  more 
undergraduate  courses  integrate  World  Wide  Web,  or  become  web-based,  learners  have  been  situated  in  a hypermedia 
intensive  learning  context.  In  a rich  hypermedia  learning  environment  such  as  web-based  or  distributed  courses,  learning  can 
be  efficient  in  that  learning  can  occur  while  students  are  separated  by  time  and  space  from  other  students  or  the  instructor. 
However,  students  may  be  misled  if  they  do  not  know  how  to  regulate  in  accessing  resources  and  seeking  information  to 
complete  learning  tasks  because  they  are  often  given  little  or  very  limited  support  from  other  learners  or  the  instructor. 
Research  showed  that  in  distributed  or  web-based  learning  environments,  different  types  of  web-based  pedagogical  tools 
facilitated  learners’  enactment  of  different  self-regulation  processes  (Kitsantas  & Dabbah,  2005). 

The  findings  from  the  studies  carried  out  by  Dabbagh  & Kitsantas  (2005)  and  Choi  & Turgeon  (2005) 
implied  that  teachers  may  decide  what  specific  type  of  web-based  pedagogical  tools  to  include  and  how  to  provide  it, 
in  accordance  to  the  learning  goals,  the  nature  of  assignments,  and  the  aspects  of  self-regulation  targeted  to  promote. 

Also,  the  result  from  the  qualitative  analysis  implied  that  when  engaging  students  in  problem  solving  tasks  in 
distributed  learning  courses,  content  delivery  tools  should  be  used  to  provide  a variety  examples  or  sample  solutions 
to  enhance  students’  self  regulatory  learning.  The  analysis  also  indicated  that  content  creation  and  delivery  tools, 
especially  the  reading,  resources,  and  assignment/rubrics  features,  were  useful  in  scaffolding  help  seeking  and  self 
evaluation  while  completing  assignments  involving  dialogical  learning  tasks.  Additionally  it  appeared  to  be 
important  that  when  engaging  students  in  articulation  and  refection  required  in  on-line  discussion,  content  and 
delivery  tools  should  provide  relevant  resources  and  also  allow  an  online  access  to  the  readings.  Providing  students 
a clear  rubric  of  how  they  will  be  evaluated  it  was  also  indicated  as  important  in  scaffolding  the  SRL  process  of  self- 
evaluation. 

A bit  differently  from  the  Dabbagh  & Kitsantas ’s  study,  which  is  conducted  in  a blended  mode  of  learning 
context  including  face  to  face  interactions,  Choi  and  colleagues(2005)  conducted  a study  in  a totally  on-line  learning 
setting.  It  was  revealed  that  the  fixed  (as  apposed  to  dynamic)  scaffolds  for  peer  questioning  were  useful  to  increase 
the  frequency  of  student  questioning  behavior  during  on-line  discussion.  However  the  guidance  did  not  improved 
the  quality  of  questions.  The  findings  suggested  that  further  studies  may  want  to  focus  on  the  quality  improvement 
of  peer-generated  questions  using  dynamic  and  adaptive  scaffolding.  How  prior  knowledge,  metacognition,  task 
complexity,  and  scaffolding  type  interact  in  on-line  learning  contexts  was  considered  to  be  an  important  future 
research  task . 

Finally,  it  is  notable  that  the  findings  from  the  studies  reviewed  in  this  paper  are  limited  in  being 
generalized  to  all  type  of  hypermedia  learning  contexts.  The  findings  are  considered  to  be  context  specific  in  terms 
of  the  degree  of  task  difficulty,  the  level  of  linearity  or  non-linearity  of  hypermedia,  the  type  of  support  ( personal  or 
non-personal,  and  fixed  or  dynamic),  and  the  type  of  delivery  mode  (on-line,  off-line,  or  both).  Therefore  an 
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application  of  the  findings  should  be  cautiously  made  considering  the  details  of  the  educational  context  in  which  the 
referred  study  is  situated.  Another  limitation  was  addressed  from  those  studies  that  examined  the  association 
between  navigational  behaviors  through  hypermedia  and  learning.  All  studies  included  could  not  highlight  the 
influence  of  learners’  prior  content  knowledge  on  hypermedia  and  self-regulation.  Navigational  behaviors 
determined  in  those  studies  might  be  mediated  by  learners’  semantic  organization  of  prior  knowledge  about  the 
content  domain  of  interest.  Some  researchers  (e.g.,  Chen  & Rada,  1996)  suggested  that  spatial  ability  may  be 
predictive  of  learning  from  hypermedia.  Further  studies  may  want  to  investigate  how  personal  variables  such  as 
prior  knowledge  and  spatial  ability  relate  to  navigational  behaviors  and  self-regulated  learning. 
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